Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> datanode tuning


Copy link to this message
-
Re: datanode tuning
Thanks Ravi. The number of nodes isn't a lot but the size is rather large.
Each data node has about 14-16T (560-640T).

For the datanode block scanner, how can increase its "Current scan rate
limit KBps" ?
On Sun, Oct 6, 2013 at 11:09 PM, Ravi Prakash <[EMAIL PROTECTED]> wrote:

> Please look at dfs.heartbeat.interval and
> dfs.namenode.heartbeat.recheck-interval
>
> 40 datanodes is not a large cluster IMHO and the Namenode is capable of
> managing 100 times more datanodes.
>
>
>
>
> ________________________________
>  From: Rita <[EMAIL PROTECTED]>
> To: "<[EMAIL PROTECTED]>" <[EMAIL PROTECTED]>
> Sent: Sunday, October 6, 2013 9:49 AM
> Subject: datanode tuning
>
>
> I would like my 40 data nodes to aggressively report to namenode if they
> are alive or not therefore I think I need to change these params
>
> dfs.block.access.token.lifetime : Default is 600 seconds. Can I decrease
> this to 60?
>
>
> dfs.block.access.key.update.interval: Default is 600 seconds. Can I
> decrease this to 60?
>
> Also, what are some other turnings people do for datanodes in a relatively
> large cluster?
>
>
>
> --
> --- Get your facts first, then you can distort them as you please.--
>

--
--- Get your facts first, then you can distort them as you please.--
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB