Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> datanode tuning


Copy link to this message
-
Re: datanode tuning
Please look at dfs.heartbeat.interval and dfs.namenode.heartbeat.recheck-interval

40 datanodes is not a large cluster IMHO and the Namenode is capable of managing 100 times more datanodes.
________________________________
 From: Rita <[EMAIL PROTECTED]>
To: "<[EMAIL PROTECTED]>" <[EMAIL PROTECTED]>
Sent: Sunday, October 6, 2013 9:49 AM
Subject: datanode tuning
 

I would like my 40 data nodes to aggressively report to namenode if they
are alive or not therefore I think I need to change these params

dfs.block.access.token.lifetime : Default is 600 seconds. Can I decrease
this to 60?
dfs.block.access.key.update.interval: Default is 600 seconds. Can I
decrease this to 60?

Also, what are some other turnings people do for datanodes in a relatively
large cluster?

--
--- Get your facts first, then you can distort them as you please.--
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB