Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> when Standby Namenode is doing checkpoint, the Active NameNode is slow.


Copy link to this message
-
Re: when Standby Namenode is doing checkpoint, the Active NameNode is slow.
Hi,
Looks like you got some pace, did you also tried with compression
parameter? I think you will get more optimization with it. Also file
transfer speed depends on our network bandwidth between PNN/SNN and network
traffic b/w nodes.What's your network conf?
Thanks

On Wed, Aug 14, 2013 at 11:39 AM, lei liu <[EMAIL PROTECTED]> wrote:

>  I set the "dfs.image.transfer.bandwidthPerSec." to 50M, and the
> performance is below:
>
> 2013-08-14 12:32:33,079 INFO my.EditLogPerformance: totalCount:1342440
> speed:1111
> 2013-08-14 12:32:43,082 INFO my.EditLogPerformance: totalCount:1363338
> speed:1044
> 2013-08-14 12:32:53,085 INFO my.EditLogPerformance: totalCount:1385526
> speed:1109
> *2013-08-14 12:33:03,087 INFO my.EditLogPerformance: totalCount:1396324
> speed:539*
> *2013-08-14 12:33:13,090 INFO my.EditLogPerformance: totalCount:1406232
> speed:495
> 2013-08-14 12:33:23,093 INFO my.EditLogPerformance: totalCount:1415006
> speed:438
> 2013-08-14 12:33:33,096 INFO my.EditLogPerformance: totalCount:1423952
> speed:447*
> *2013-08-14 12:33:43,099 INFO my.EditLogPerformance: totalCount:1437256
> speed:665*
> 2013-08-14 12:33:53,102 INFO my.EditLogPerformance: totalCount:1458378
> speed:1056
> 2013-08-14 12:34:03,106 INFO my.EditLogPerformance: totalCount:1479338
> speed:1048
> 2013-08-14 12:34:13,108 INFO my.EditLogPerformance: totalCount:1500400
> speed:1053
> 2013-08-14 12:34:23,111 INFO my.EditLogPerformance: totalCount:1521252
> speed:1042
> 2013-08-14 12:34:33,114 INFO my.EditLogPerformance: totalCount:1542286
> speed:1051
> 2013-08-14 12:34:43,117 INFO my.EditLogPerformance: totalCount:1562956
> speed:1033
> 2013-08-14 12:34:53,120 INFO my.EditLogPerformance: totalCount:1583804
> speed:1042
> 2013-08-14 12:35:03,123 INFO my.EditLogPerformance: totalCount:1606558
> speed:1137
> 2013-08-14 12:35:13,126 INFO my.EditLogPerformance: totalCount:1627980
> speed:1071
> 2013-08-14 12:35:23,129 INFO my.EditLogPerformance: totalCount:1650642
> speed:1133
> 2013-08-14 12:35:33,132 INFO my.EditLogPerformance: totalCount:1672806
> speed:1108
> 2013-08-14 12:35:43,134 INFO my.EditLogPerformance: totalCount:1693940
> speed:1056
> 2013-08-14 12:35:53,137 INFO my.EditLogPerformance: totalCount:1715430
> speed:1074
> 2013-08-14 12:36:03,140 INFO my.EditLogPerformance: totalCount:1737940
> speed:1125
> 2013-08-14 12:36:13,143 INFO my.EditLogPerformance: totalCount:1760094
> speed:1107
> 2013-08-14 12:36:23,146 INFO my.EditLogPerformance: totalCount:1781646
> speed:1077
> 2013-08-14 12:36:33,149 INFO my.EditLogPerformance: totalCount:1802230
> speed:1029
> 2013-08-14 12:36:43,152 INFO my.EditLogPerformance: totalCount:1824132
> speed:1095
> 2013-08-14 12:36:53,155 INFO my.EditLogPerformance: totalCount:1846778
> speed:1132
> 2013-08-14 12:37:03,158 INFO my.EditLogPerformance: totalCount:1868956
> speed:1108
> 2013-08-14 12:37:13,161 INFO my.EditLogPerformance: totalCount:1888556
> speed:980
> 2013-08-14 12:37:23,164 INFO my.EditLogPerformance: totalCount:1910512
> speed:1097
> 2013-08-14 12:37:33,167 INFO my.EditLogPerformance: totalCount:1932240
> speed:1086
> 2013-08-14 12:37:43,170 INFO my.EditLogPerformance: totalCount:1954226
> speed:1099
> 2013-08-14 12:37:53,173 INFO my.EditLogPerformance: totalCount:1974706
> speed:1024
> 2013-08-14 12:38:03,176 INFO my.EditLogPerformance: totalCount:1993906
> speed:960
> 2013-08-14 12:38:13,179 INFO my.EditLogPerformance: totalCount:2014172
> speed:1013
> 2013-08-14 12:38:23,182 INFO my.EditLogPerformance: totalCount:2036130
> speed:1097
> 2013-08-14 12:38:33,184 INFO my.EditLogPerformance: totalCount:2057848
> speed:1085
> 2013-08-14 12:38:43,187 INFO my.EditLogPerformance: totalCount:2078834
> speed:1049
> 2013-08-14 12:38:53,190 INFO my.EditLogPerformance: totalCount:2095616
> speed:839
> *2013-08-14 12:39:03,193 INFO my.EditLogPerformance: totalCount:2104896
> speed:464
> 2013-08-14 12:39:13,196 INFO my.EditLogPerformance: totalCount:2114572
> speed:483
> 2013-08-14 12:39:23,199 INFO my.EditLogPerformance: totalCount:2123512
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB