Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> map stucks at 99.99%


Copy link to this message
-
Re: map stucks at 99.99%
- Check the box on which the task is running, is it under heavy load ?
Is there high amount of I/O wait ?
CPU, very warm load average: 47.47, 48.56, 49.00
I/O, chill on io 0.1x % on iowait, less than 20 tps, rarely upto
100tps, on 10 disks jbod.
- You could check the task logs and see if they say anything about
what is going wrong ?
I would say no.. pretty much all of them is INFO

- Did the task get pre-empted to other task trackers ? If yes, is it
stuck at the same spot on those ?
Nope.

- What kind of work are you doing in the mapper ? Just reading from
HDFS and compute something or reading/writing from HBase ?
HDFS + compute, R/W
Absolutely no HBase.

Would jstack, jmap be any useful ?
> - You could check the task logs and see if they say anything about what is
> going wrong ?
> - Did the task get pre-empted to other task trackers ? If yes, is it stuck
> at the same spot on those ?
> - What kind of work are you doing in the mapper ? Just reading from HDFS and
> compute something or reading/writing from HBase ?

On Thu, Feb 28, 2013 at 12:25 PM, Viral Bajaria <[EMAIL PROTECTED]> wrote:
> You could start off doing the following:
>
> - Check the box on which the task is running, is it under heavy load ? Is
> there high amount of I/O wait ?
> - You could check the task logs and see if they say anything about what is
> going wrong ?
> - Did the task get pre-empted to other task trackers ? If yes, is it stuck
> at the same spot on those ?
> - What kind of work are you doing in the mapper ? Just reading from HDFS and
> compute something or reading/writing from HBase ?
>
> Thanks,
> Viral
>
> On Thu, Feb 28, 2013 at 12:06 PM, Patai Sangbutsarakum
> <[EMAIL PROTECTED]> wrote:
>>
>> Hadoopers!!
>>
>> Need input from you guys,
>> i am looking at a critical job in production. it stucks at 99.99% in
>> map phrase for much longer than it used to be..
>>
>> what to do to debug what is going on with those map why it is not pass
>> through
>> even though tasks and task attempts saying 100% progress but there is
>> not finish time...
>>
>> Please suggest
>> Patai
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB