14-16 Tb is perhaps a big node. Even then the scalability limits of the Namenode in your case would depend on how many files (more accurately how many blocks) there are on HDFS.
In any case, if you want the datanodes to be marked dead quickly when their heartbeats are lost, you should reduce the two parameters I told you about.
The datanode block scanner is unfortunately hard coded to use a maximum of 8Mb/s and a minimum of 1 Mb/s. The only thing you can change is dfs.datanode.scan.period.hours
From: Rita <[EMAIL PROTECTED]>
To: "<[EMAIL PROTECTED]>" <[EMAIL PROTECTED]>; Ravi Prakash <[EMAIL PROTECTED]>
Sent: Monday, October 7, 2013 5:55 AM
Subject: Re: datanode tuning
Thanks Ravi. The number of nodes isn't a lot but the size is rather large.
Each data node has about 14-16T (560-640T).
For the datanode block scanner, how can increase its "Current scan rate
limit KBps" ?
On Sun, Oct 6, 2013 at 11:09 PM, Ravi Prakash <[EMAIL PROTECTED]> wrote:
> Please look at dfs.heartbeat.interval and
> 40 datanodes is not a large cluster IMHO and the Namenode is capable of
> managing 100 times more datanodes.
> From: Rita <[EMAIL PROTECTED]>
> To: "<[EMAIL PROTECTED]>" <[EMAIL PROTECTED]>
> Sent: Sunday, October 6, 2013 9:49 AM
> Subject: datanode tuning
> I would like my 40 data nodes to aggressively report to namenode if they
> are alive or not therefore I think I need to change these params
> dfs.block.access.token.lifetime : Default is 600 seconds. Can I decrease
> this to 60?
> dfs.block.access.key.update.interval: Default is 600 seconds. Can I
> decrease this to 60?
> Also, what are some other turnings people do for datanodes in a relatively
> large cluster?
> --- Get your facts first, then you can distort them as you please.--
--- Get your facts first, then you can distort them as you please.--