Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Reduce task hang[EMERGENCE]


Copy link to this message
-
Re: Reduce task hang[EMERGENCE]
Add addtional:

Our MR version is 1.2.1, not 1.0.4

There is no useful information in the JT log.
On Fri, Jan 3, 2014 at 12:20 PM, Azuryy Yu <[EMAIL PROTECTED]> wrote:

> Hi,
>
> Our prod cluster met some issues recently,
> All map tasks finished successfully, but reduce task hanged.
>
> but It's not happened on all TaskTrackers, only sometimes. we used
> mapred-1.0.4
>
> There is "0.0% reduce > copy >" forever until kill task manually.
>
> reduce logs on the TaskTracker:
> hadoop-hadoop-tasktracker-10-200-91-186.out:14/01/03 06:13:57 INFO
> mapred.TaskTracker: JVM with ID: jvm_201312201200_34795_r_-365330778 given
> task: attempt_201312201200_34795_r_000000_0
> hadoop-hadoop-tasktracker-10-200-91-186.out:14/01/03 06:14:04 INFO
> mapred.TaskTracker: attempt_201312201200_34795_r_000000_0 0.0% reduce >
> copy >
> hadoop-hadoop-tasktracker-10-200-91-186.out:14/01/03 06:14:08 INFO
> mapred.TaskTracker: attempt_201312201200_34795_r_000000_0 0.0% reduce >
> copy >
> hadoop-hadoop-tasktracker-10-200-91-186.out:14/01/03 06:14:14 INFO
> mapred.TaskTracker: attempt_201312201200_34795_r_000000_0 0.0% reduce >
> copy >
> hadoop-hadoop-tasktracker-10-200-91-186.out:14/01/03 06:14:17 INFO
> mapred.TaskTracker: attempt_201312201200_34795_r_000000_0 0.0% reduce >
> copy >
> hadoop-hadoop-tasktracker-10-200-91-186.out:14/01/03 06:14:23 INFO
> mapred.TaskTracker: attempt_201312201200_34795_r_000000_0 0.0% reduce >
> copy >
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB