Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # user >> Re: when Standby Namenode is doing checkpoint, the Active NameNode is slow.


+
lei liu 2013-08-13, 06:22
+
Jitendra Yadav 2013-08-13, 07:42
Copy link to this message
-
Re: when Standby Namenode is doing checkpoint, the Active NameNode is slow.
Perhaps turning on fsimage compression may help. See
documentation of dfs.image.compress at
http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/hdfs-default.xml.

You can also try to throttle the bandwidth it uses via
dfs.image.transfer.bandwidthPerSec.

On Tue, Aug 13, 2013 at 3:45 PM, lei liu <[EMAIL PROTECTED]> wrote:
> I write one programm to test NameNode performance. Please see the
> EditLogPerformance.java
>
> I use 60 threads to execute the EditLogPerformance.javacode,  the testing
> result is below content:
> 2013-08-13 17:43:01,479 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10392810  speed:1055
> 2013-08-13 17:43:11,482 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10407310  speed:725
> 2013-08-13 17:43:21,484 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10407358  speed:2
> 2013-08-13 17:43:31,487 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10407490  speed:6
> 2013-08-13 17:43:41,490 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10407624  speed:6
> 2013-08-13 17:43:51,493 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10408690  speed:53
> 2013-08-13 17:44:01,496 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10422220  speed:676
> 2013-08-13 17:44:11,499 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10445216  speed:1149
> 2013-08-13 17:44:21,502 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10465166  speed:997
> 2013-08-13 17:44:31,505 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10486614  speed:1072
> 2013-08-13 17:44:41,508 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10506778  speed:1008
> 2013-08-13 17:44:51,511 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10526660  speed:994
> 2013-08-13 17:45:01,514 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10548092  speed:1071
> 2013-08-13 17:45:11,517 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10569892  speed:1090
> 2013-08-13 17:45:21,520 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10593296  speed:1170
> 2013-08-13 17:45:31,523 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10614478  speed:1059
> 2013-08-13 17:45:41,526 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10636006  speed:1076
> 2013-08-13 17:45:51,529 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10656430  speed:1021
> 2013-08-13 17:46:01,532 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10677774  speed:1067
> 2013-08-13 17:46:11,534 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10699096  speed:1066
> 2013-08-13 17:46:21,537 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10720970  speed:1093
> 2013-08-13 17:46:31,540 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10741432  speed:1023
> 2013-08-13 17:46:41,543 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10760854  speed:971
> 2013-08-13 17:46:51,546 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10781680  speed:1041
> 2013-08-13 17:47:01,549 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10802302  speed:1031
> 2013-08-13 17:47:11,552 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10823888  speed:1079
> 2013-08-13 17:47:21,555 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10845276  speed:1069
> 2013-08-13 17:47:31,558 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10865470  speed:1009
> 2013-08-13 17:47:41,561 INFO  my.EditLogPerformance

Harsh J
+
Jitendra Yadav 2013-08-13, 11:24
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB