Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Re: mapreduce.jobtracker.expire.trackers.interval no effect


Copy link to this message
-
Re: mapreduce.jobtracker.expire.trackers.interval no effect
I did a small test, and I a setting
mapred.tasktracker.expiry.interval=60000 worked
for me (TT became considered as lost after around 66 seconds).

Can the formula be: mapred.tasktracker.expiry.interval + 2 *
some-heartbeat-interval-that-is-3-sec-by-default?
Otherwise, is the 6 sec some kind of time needed to make a decision to
consider TT as lost?
2013/12/3 Hansi Klose <[EMAIL PROTECTED]>

> I forget to say that we use Cloudera 2.0.0-mr1-cdh4.2.0
>
> > Gesendet: Dienstag, 03. Dezember 2013 um 17:38 Uhr
> > Von: "Hansi Klose" <[EMAIL PROTECTED]>
> > An: [EMAIL PROTECTED]
> > Betreff: mapreduce.jobtracker.expire.trackers.interval no effect
> >
> > Hi,
> >
> > we want to set the heartbeat timout for a tasktracker.
> >
> > If the tasktracker does not send heartbeats for 60 seconds he should
> > be marked as lost.
> >
> > I found the parameter mapreduce.jobtracker.expire.trackers.interval
> > which sounds right to me.
> >
> > I set
> >
> > <property>
> > <name>mapreduce.jobtracker.expire.trackers.interval</name>
> > <value>60000</value>
> > </property>
> >
> > in the mapred-site.xml on all servers and restarted the jobtracker and
> all tasktrackers.
> >
> > I started a benchmark "hadoop jar hadoop-examples.jar randomwriter rand"
> and every tasktracker gets 2 jobs.
> > It is a small test environment.
> >
> > On one tasktracker i stopped the network. On the jobtracker i could see
> the "Seconds since heartbeat"
> > increasing. But after 60 seconds the tasktracker was still in the
> overview.
> > Even in the log of the jobtracker I found nothing.
> >
> > After over 600 seconds i found the message
> > org.apache.hadoop.mapred.JobTracker: Lost tracker .....
> > And the tasktracker wasn't shown any more on the jobtracker.
> >
> > Isn't this the right setting?
> >
> > Regards Hansi
> >
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB