Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - datanode tuning

Copy link to this message
Re: datanode tuning
Ravi Prakash 2013-10-07, 03:09
Please look at dfs.heartbeat.interval and dfs.namenode.heartbeat.recheck-interval

40 datanodes is not a large cluster IMHO and the Namenode is capable of managing 100 times more datanodes.
Sent: Sunday, October 6, 2013 9:49 AM
Subject: datanode tuning

I would like my 40 data nodes to aggressively report to namenode if they
are alive or not therefore I think I need to change these params

dfs.block.access.token.lifetime : Default is 600 seconds. Can I decrease
this to 60?
dfs.block.access.key.update.interval: Default is 600 seconds. Can I
decrease this to 60?

Also, what are some other turnings people do for datanodes in a relatively
large cluster?

--- Get your facts first, then you can distort them as you please.--