Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Stuck MR job


Copy link to this message
-
Re: Stuck MR job
Vidhya,

> Hi
>  This looks like a trivial problem but would be glad if someone can help..
>
>  I have been trying to run a m-r job on my cluster. I had modified my configs (primarily reduced the heap sizes for the task tracker and the data nodes) and restarted my hadoop cluster and the job won't start: The job tracker keeps spewing out logs of the form:
>
> 2010-06-23 08:36:28,350 INFO org.apache.hadoop.mapreduce.server.jobtracker.TaskTracker: tracker_b3060080.yst.yahoo.net:localhost/127.0.0.1:55818: Reserved 2 MAP slots for job_201006230709_0008
> 2010-06-23 08:36:28,350 INFO org.apache.hadoop.mapreduce.server.jobtracker.TaskTracker: tracker_b3060114.yst.yahoo.net:localhost/127.0.0.1:55508: Reserved 2 MAP slots for job_201006230709_0008
> 2010-06-23 08:36:28,353 INFO org.apache.hadoop.mapreduce.server.jobtracker.TaskTracker: tracker_b3060083.yst.yahoo.net:localhost/127.0.0.1:55675: Reserved 2 MAP slots for job_201006230709_0008
> 2010-06-23 08:36:28,354 INFO org.apache.hadoop.mapreduce.server.jobtracker.TaskTracker: tracker_b3060054.yst.yahoo.net:localhost/127.0.0.1:59455: Reserved 2 MAP slots for job_201006230709_0008
> 2010-06-23 08:36:31,352 INFO org.apache.hadoop.mapreduce.server.jobtracker.TaskTracker: tracker_b3060059.yst.yahoo.net:localhost/127.0.0.1:55771: Reserved 2 MAP slots for job_201006230709_0008

This looks like you are using the Capacity scheduler with high memory
job configuration. Did you set any specific memory parameters ? Also,
are there no jobs in the cluster other than this one ? Can you give
the version of Hadoop being used, and configuration of memory related
parameters ?

>
> And the cluster shows no activity of the job progressing: (% complete is 0 in the map jobs).. I see no errors/activity in tasktrackers..
>
> I would appreciate it if someone helps me with this one.
> Thank you
> Vidhya
>