Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # user - Re: mapreduce.jobtracker.expire.trackers.interval no effect


Copy link to this message
-
Re: mapreduce.jobtracker.expire.trackers.interval no effect
Adam Kawa 2013-12-03, 18:23
I did a small test, and I a setting
mapred.tasktracker.expiry.interval=60000 worked
for me (TT became considered as lost after around 66 seconds).

Can the formula be: mapred.tasktracker.expiry.interval + 2 *
some-heartbeat-interval-that-is-3-sec-by-default?
Otherwise, is the 6 sec some kind of time needed to make a decision to
consider TT as lost?
2013/12/3 Hansi Klose <[EMAIL PROTECTED]>

> I forget to say that we use Cloudera 2.0.0-mr1-cdh4.2.0
>
> > Gesendet: Dienstag, 03. Dezember 2013 um 17:38 Uhr
> > Von: "Hansi Klose" <[EMAIL PROTECTED]>
> > An: [EMAIL PROTECTED]
> > Betreff: mapreduce.jobtracker.expire.trackers.interval no effect
> >
> > Hi,
> >
> > we want to set the heartbeat timout for a tasktracker.
> >
> > If the tasktracker does not send heartbeats for 60 seconds he should
> > be marked as lost.
> >
> > I found the parameter mapreduce.jobtracker.expire.trackers.interval
> > which sounds right to me.
> >
> > I set
> >
> > <property>
> > <name>mapreduce.jobtracker.expire.trackers.interval</name>
> > <value>60000</value>
> > </property>
> >
> > in the mapred-site.xml on all servers and restarted the jobtracker and
> all tasktrackers.
> >
> > I started a benchmark "hadoop jar hadoop-examples.jar randomwriter rand"
> and every tasktracker gets 2 jobs.
> > It is a small test environment.
> >
> > On one tasktracker i stopped the network. On the jobtracker i could see
> the "Seconds since heartbeat"
> > increasing. But after 60 seconds the tasktracker was still in the
> overview.
> > Even in the log of the jobtracker I found nothing.
> >
> > After over 600 seconds i found the message
> > org.apache.hadoop.mapred.JobTracker: Lost tracker .....
> > And the tasktracker wasn't shown any more on the jobtracker.
> >
> > Isn't this the right setting?
> >
> > Regards Hansi
> >
>