Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # user >> Re: when Standby Namenode is doing checkpoint, the Active NameNode is slow.


+
lei liu 2013-08-13, 06:22
+
Jitendra Yadav 2013-08-13, 07:42
Copy link to this message
-
Re: when Standby Namenode is doing checkpoint, the Active NameNode is slow.
Perhaps turning on fsimage compression may help. See
documentation of dfs.image.compress at
http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/hdfs-default.xml.

You can also try to throttle the bandwidth it uses via
dfs.image.transfer.bandwidthPerSec.

On Tue, Aug 13, 2013 at 3:45 PM, lei liu <[EMAIL PROTECTED]> wrote:
> I write one programm to test NameNode performance. Please see the
> EditLogPerformance.java
>
> I use 60 threads to execute the EditLogPerformance.javacode,  the testing
> result is below content:
> 2013-08-13 17:43:01,479 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10392810  speed:1055
> 2013-08-13 17:43:11,482 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10407310  speed:725
> 2013-08-13 17:43:21,484 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10407358  speed:2
> 2013-08-13 17:43:31,487 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10407490  speed:6
> 2013-08-13 17:43:41,490 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10407624  speed:6
> 2013-08-13 17:43:51,493 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10408690  speed:53
> 2013-08-13 17:44:01,496 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10422220  speed:676
> 2013-08-13 17:44:11,499 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10445216  speed:1149
> 2013-08-13 17:44:21,502 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10465166  speed:997
> 2013-08-13 17:44:31,505 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10486614  speed:1072
> 2013-08-13 17:44:41,508 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10506778  speed:1008
> 2013-08-13 17:44:51,511 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10526660  speed:994
> 2013-08-13 17:45:01,514 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10548092  speed:1071
> 2013-08-13 17:45:11,517 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10569892  speed:1090
> 2013-08-13 17:45:21,520 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10593296  speed:1170
> 2013-08-13 17:45:31,523 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10614478  speed:1059
> 2013-08-13 17:45:41,526 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10636006  speed:1076
> 2013-08-13 17:45:51,529 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10656430  speed:1021
> 2013-08-13 17:46:01,532 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10677774  speed:1067
> 2013-08-13 17:46:11,534 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10699096  speed:1066
> 2013-08-13 17:46:21,537 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10720970  speed:1093
> 2013-08-13 17:46:31,540 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10741432  speed:1023
> 2013-08-13 17:46:41,543 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10760854  speed:971
> 2013-08-13 17:46:51,546 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10781680  speed:1041
> 2013-08-13 17:47:01,549 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10802302  speed:1031
> 2013-08-13 17:47:11,552 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10823888  speed:1079
> 2013-08-13 17:47:21,555 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10845276  speed:1069
> 2013-08-13 17:47:31,558 INFO  my.EditLogPerformance
> (EditLogPerformance.java:run(37)) - totalCount:10865470  speed:1009
> 2013-08-13 17:47:41,561 INFO  my.EditLogPerformance

Harsh J
+
Jitendra Yadav 2013-08-13, 11:24