Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> datanode tuning


+
Rita 2013-10-06, 14:49
+
Ravi Prakash 2013-10-07, 03:09
Copy link to this message
-
Re: datanode tuning
Thanks Ravi. The number of nodes isn't a lot but the size is rather large.
Each data node has about 14-16T (560-640T).

For the datanode block scanner, how can increase its "Current scan rate
limit KBps" ?
On Sun, Oct 6, 2013 at 11:09 PM, Ravi Prakash <[EMAIL PROTECTED]> wrote:

> Please look at dfs.heartbeat.interval and
> dfs.namenode.heartbeat.recheck-interval
>
> 40 datanodes is not a large cluster IMHO and the Namenode is capable of
> managing 100 times more datanodes.
>
>
>
>
> ________________________________
>  From: Rita <[EMAIL PROTECTED]>
> To: "<[EMAIL PROTECTED]>" <[EMAIL PROTECTED]>
> Sent: Sunday, October 6, 2013 9:49 AM
> Subject: datanode tuning
>
>
> I would like my 40 data nodes to aggressively report to namenode if they
> are alive or not therefore I think I need to change these params
>
> dfs.block.access.token.lifetime : Default is 600 seconds. Can I decrease
> this to 60?
>
>
> dfs.block.access.key.update.interval: Default is 600 seconds. Can I
> decrease this to 60?
>
> Also, what are some other turnings people do for datanodes in a relatively
> large cluster?
>
>
>
> --
> --- Get your facts first, then you can distort them as you please.--
>

--
--- Get your facts first, then you can distort them as you please.--
+
Ravi Prakash 2013-10-07, 14:50
+
Rita 2013-10-07, 21:09