Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> heartbeat and timeout question


Copy link to this message
-
heartbeat and timeout question
Hello Hadoopers,
I am going to migrate production racks of datanodes/tasktrackers into new
core switches. Rack awareness is in place for long time. I am looking for
the way to mitigate recopying blocks of datanodes in the rack that is being
move (when it become dead nodes), and shifting of running tasks in those
tasktrackers to other machines.

One approach, that i can thinking of is playing with heartbeat of both
datanode and tasktracker to make it extra long like 15 minutes, so namenode
and jobtracker are more forgiving to those nodes (that is being moved).
however, network operation that need to be done to flip the switch should
be around couple minutes per rack.

Possible alternatives are more than welcome.

Thanks in advnace,
P

btw, the cluster is on cdh3u4 (0.20 branch)
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB