Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Stuck MR job


Copy link to this message
-
Re: Stuck MR job
Vidhya,

>>> Can you give the version of Hadoop being used, and configuration of memory related parameters ?
> I did reduce the heap sizes since I wanted to allocate more for Hbase (running in tandem).
> Total mem available: 8 gigs..
>  I allocated 5 gigs for Hbase..
> As it stands, right now:
> MR jobs: 512M each
> 2 max MR jobs
> DN and Task trackers: 1 Gig each
>

Unfortunately, I still need more information:

- What is the version/distribution of Hadoop ? Are you using Capacity
scheduler as the scheduler ?
I would like to know the values set for some specific configuration
variables. Unfortunately, the names of those variables have changed
from 20 to 21 and trunk. Hence, I need to know the version to specify
which ones to look up for.

Thanks
Hemanth

> Vidhya
>
> On 6/23/10 3:16 AM, "Hemanth Yamijala" <[EMAIL PROTECTED]> wrote:
>
> Vidhya,
>
>> Hi
>>  This looks like a trivial problem but would be glad if someone can help..
>>
>>  I have been trying to run a m-r job on my cluster. I had modified my configs (primarily reduced the heap sizes for the task tracker and the data nodes) and restarted my hadoop cluster and the job won't start: The job tracker keeps spewing out logs of the form:
>>
>> 2010-06-23 08:36:28,350 INFO org.apache.hadoop.mapreduce.server.jobtracker.TaskTracker: tracker_b3060080.yst.yahoo.net:localhost/127.0.0.1:55818: Reserved 2 MAP slots for job_201006230709_0008
>> 2010-06-23 08:36:28,350 INFO org.apache.hadoop.mapreduce.server.jobtracker.TaskTracker: tracker_b3060114.yst.yahoo.net:localhost/127.0.0.1:55508: Reserved 2 MAP slots for job_201006230709_0008
>> 2010-06-23 08:36:28,353 INFO org.apache.hadoop.mapreduce.server.jobtracker.TaskTracker: tracker_b3060083.yst.yahoo.net:localhost/127.0.0.1:55675: Reserved 2 MAP slots for job_201006230709_0008
>> 2010-06-23 08:36:28,354 INFO org.apache.hadoop.mapreduce.server.jobtracker.TaskTracker: tracker_b3060054.yst.yahoo.net:localhost/127.0.0.1:59455: Reserved 2 MAP slots for job_201006230709_0008
>> 2010-06-23 08:36:31,352 INFO org.apache.hadoop.mapreduce.server.jobtracker.TaskTracker: tracker_b3060059.yst.yahoo.net:localhost/127.0.0.1:55771: Reserved 2 MAP slots for job_201006230709_0008
>
> This looks like you are using the Capacity scheduler with high memory
> job configuration. Did you set any specific memory parameters ? Also,
> are there no jobs in the cluster other than this one ? Can you give
> the version of Hadoop being used, and configuration of memory related
> parameters ?
>
>>
>> And the cluster shows no activity of the job progressing: (% complete is 0 in the map jobs).. I see no errors/activity in tasktrackers..
>>
>> I would appreciate it if someone helps me with this one.
>> Thank you
>> Vidhya
>>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB