Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> YARN Pi example job stuck at 0%(No MR tasks are started by ResourceManager)


+
anil gupta 2012-07-27, 18:23
+
Rahul Jain 2012-07-30, 23:26
+
anil gupta 2012-07-30, 23:56
+
abhiTowson cal 2012-07-31, 02:30
+
anil gupta 2012-07-31, 02:47
+
abhiTowson cal 2012-07-31, 03:12
+
anil gupta 2012-07-31, 03:51
+
abhiTowson cal 2012-07-31, 04:21
+
anil gupta 2012-07-31, 18:26
+
anil gupta 2012-08-02, 23:25
Copy link to this message
-
Re: YARN Pi example job stuck at 0%(No MR tasks are started by ResourceManager)
Yes, do ensure that JAVA_HOME is properly set on all nodes through hadoop
env or the login shell.

Regarding agregated logs,  they will be found in hdfs under the directory
set through 'yarn.nodemanager.remote-app-log-dir' , a subdirectory for each
job.

-Rahul

On Mon, Jul 30, 2012 at 8:12 PM, abhiTowson cal
<[EMAIL PROTECTED]>wrote:

> Hi anil,
>
> Adding property resolved issue for me, and i also made this change
>
> vim hadoop-env.sh
>
> export JAVA_HOME=/usr/lib/java-1.6.0/jdk1.6.0_33
> if [ "$JAVA_HOME" != "" ]; then
>   #echo "run java in $JAVA_HOME"
>   JAVA_HOME=$JAVA_HOME
> fi
>
> if [ "$JAVA_HOME" = "" ]; then
>   echo "Error: JAVA_HOME is not set."
>   exit 1
> fi
>
> JAVA=$JAVA_HOME/bin/java
> JAVA_HEAP_MAX=-Xmx1000m
>
> Regards
> Abhishek
>
>
> On Mon, Jul 30, 2012 at 10:47 PM, anil gupta <[EMAIL PROTECTED]>
> wrote:
> > Hi Abhishek,
> >
> > Did you mean that adding yarn.resourcemanager.resource-tracker.address
> > along with yarn.log-aggregation-enable in my configuration will resolve
> the
> > problem in which map-reduce job fails at 0% with the following error: In
> > the web page of
> >
> http://data-node:8042/node/containerlogs/container_1343687008058_0003_01_000001/rootthe
> > page says:
> > Failed redirect for container_1343687008058_0003_01_000001  Failed while
> > trying to construct the redirect url to the log server. Log Server url
> may
> > not be configured. Unknown container. Container either has not started or
> > has already completed or doesn't belong to this node at all.
> > Please let me know.
> >
> > Thanks,
> > Anil Gupta
> >
> > On Mon, Jul 30, 2012 at 7:30 PM, abhiTowson cal
> > <[EMAIL PROTECTED]>wrote:
> >
> >> hi anil,
> >>
> >> Adding these help me resolve the issue for me
> >> yarn.resourcemanager.resource-tracker.address
> >>
> >> Regards
> >> Abhishek
> >>
> >> On Mon, Jul 30, 2012 at 7:56 PM, anil gupta <[EMAIL PROTECTED]>
> wrote:
> >> > Hi Rahul,
> >> >
> >> > Thanks for your response. I can certainly enable the
> >> > yarn.log-aggregation-enable to true. But after enabling this what
> manual
> >> > steps i will have to take to run jobs. Could you please elaborate.
> >> >
> >> > Thanks,
> >> > Anil
> >> >
> >> > On Mon, Jul 30, 2012 at 4:26 PM, Rahul Jain <[EMAIL PROTECTED]> wrote:
> >> >
> >> >> The inability to look at map-reduce logs for failed logs is due to
> >> number
> >> >> of open issues in yarn; see my recent comment here:
> >> >>
> >> >>
> >>
> https://issues.apache.org/jira/browse/MAPREDUCE-4428?focusedCommentId=13412995&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13412995
> >> >>
> >> >> You can workaround this by enabling log aggregation and manually
> copying
> >> >> job logs from HDFS log location. Of course that is a painful way till
> >> the
> >> >> yarn log collection and history bugs are resolved in an upcoming
> >> release.
> >> >>
> >> >> -Rahul
> >> >>
> >> >>
> >> >> > 12/07/27 09:38:27 INFO mapred.ResourceMgrDelegate: Submitted
> >> application
> >> >> > application_1343365114818_0002 to ResourceManager at ihub-an-l1/
> >> >> > 172.31.192.151:8040
> >> >> > 12/07/27 09:38:27 INFO mapreduce.Job: The url to track the job:
> >> >> > http://ihub-an-l1:9999/proxy/application_1343365114818_0002/
> >> >> > 12/07/27 09:38:27 INFO mapreduce.Job: Running job:
> >> job_1343365114818_0002
> >> >> >
> >> >> > No Map-Reduce task are started by the cluster. I dont see any
> errors
> >> >> > anywhere in the application. Please help me in resolving this
> problem.
> >> >> >
> >> >> > Thanks,
> >> >> > Anil Gupta
> >> >> >
> >> >>
> >> >
> >> >
> >> >
> >> > --
> >> > Thanks & Regards,
> >> > Anil Gupta
> >>
> >
> >
> >
> > --
> > Thanks & Regards,
> > Anil Gupta
>
+
Harsh J 2012-07-27, 21:23
+
anil gupta 2012-07-27, 22:05
+
Harsh J 2012-07-27, 22:36
+
anil gupta 2012-07-27, 23:22
+
Harsh J 2012-07-27, 23:39
+
anil gupta 2012-07-30, 23:03
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB