Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> YARN Pi example job stuck at 0%(No MR tasks are started by ResourceManager)


+
anil gupta 2012-07-27, 18:23
+
Rahul Jain 2012-07-30, 23:26
+
anil gupta 2012-07-30, 23:56
+
abhiTowson cal 2012-07-31, 02:30
+
anil gupta 2012-07-31, 02:47
+
abhiTowson cal 2012-07-31, 03:12
+
anil gupta 2012-07-31, 03:51
+
abhiTowson cal 2012-07-31, 04:21
+
anil gupta 2012-07-31, 18:26
+
anil gupta 2012-08-02, 23:25
Copy link to this message
-
Re: YARN Pi example job stuck at 0%(No MR tasks are started by ResourceManager)
Yes, do ensure that JAVA_HOME is properly set on all nodes through hadoop
env or the login shell.

Regarding agregated logs,  they will be found in hdfs under the directory
set through 'yarn.nodemanager.remote-app-log-dir' , a subdirectory for each
job.

-Rahul

On Mon, Jul 30, 2012 at 8:12 PM, abhiTowson cal
<[EMAIL PROTECTED]>wrote:

> Hi anil,
>
> Adding property resolved issue for me, and i also made this change
>
> vim hadoop-env.sh
>
> export JAVA_HOME=/usr/lib/java-1.6.0/jdk1.6.0_33
> if [ "$JAVA_HOME" != "" ]; then
>   #echo "run java in $JAVA_HOME"
>   JAVA_HOME=$JAVA_HOME
> fi
>
> if [ "$JAVA_HOME" = "" ]; then
>   echo "Error: JAVA_HOME is not set."
>   exit 1
> fi
>
> JAVA=$JAVA_HOME/bin/java
> JAVA_HEAP_MAX=-Xmx1000m
>
> Regards
> Abhishek
>
>
> On Mon, Jul 30, 2012 at 10:47 PM, anil gupta <[EMAIL PROTECTED]>
> wrote:
> > Hi Abhishek,
> >
> > Did you mean that adding yarn.resourcemanager.resource-tracker.address
> > along with yarn.log-aggregation-enable in my configuration will resolve
> the
> > problem in which map-reduce job fails at 0% with the following error: In
> > the web page of
> >
> http://data-node:8042/node/containerlogs/container_1343687008058_0003_01_000001/rootthe
> > page says:
> > Failed redirect for container_1343687008058_0003_01_000001  Failed while
> > trying to construct the redirect url to the log server. Log Server url
> may
> > not be configured. Unknown container. Container either has not started or
> > has already completed or doesn't belong to this node at all.
> > Please let me know.
> >
> > Thanks,
> > Anil Gupta
> >
> > On Mon, Jul 30, 2012 at 7:30 PM, abhiTowson cal
> > <[EMAIL PROTECTED]>wrote:
> >
> >> hi anil,
> >>
> >> Adding these help me resolve the issue for me
> >> yarn.resourcemanager.resource-tracker.address
> >>
> >> Regards
> >> Abhishek
> >>
> >> On Mon, Jul 30, 2012 at 7:56 PM, anil gupta <[EMAIL PROTECTED]>
> wrote:
> >> > Hi Rahul,
> >> >
> >> > Thanks for your response. I can certainly enable the
> >> > yarn.log-aggregation-enable to true. But after enabling this what
> manual
> >> > steps i will have to take to run jobs. Could you please elaborate.
> >> >
> >> > Thanks,
> >> > Anil
> >> >
> >> > On Mon, Jul 30, 2012 at 4:26 PM, Rahul Jain <[EMAIL PROTECTED]> wrote:
> >> >
> >> >> The inability to look at map-reduce logs for failed logs is due to
> >> number
> >> >> of open issues in yarn; see my recent comment here:
> >> >>
> >> >>
> >>
> https://issues.apache.org/jira/browse/MAPREDUCE-4428?focusedCommentId=13412995&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13412995
> >> >>
> >> >> You can workaround this by enabling log aggregation and manually
> copying
> >> >> job logs from HDFS log location. Of course that is a painful way till
> >> the
> >> >> yarn log collection and history bugs are resolved in an upcoming
> >> release.
> >> >>
> >> >> -Rahul
> >> >>
> >> >>
> >> >> > 12/07/27 09:38:27 INFO mapred.ResourceMgrDelegate: Submitted
> >> application
> >> >> > application_1343365114818_0002 to ResourceManager at ihub-an-l1/
> >> >> > 172.31.192.151:8040
> >> >> > 12/07/27 09:38:27 INFO mapreduce.Job: The url to track the job:
> >> >> > http://ihub-an-l1:9999/proxy/application_1343365114818_0002/
> >> >> > 12/07/27 09:38:27 INFO mapreduce.Job: Running job:
> >> job_1343365114818_0002
> >> >> >
> >> >> > No Map-Reduce task are started by the cluster. I dont see any
> errors
> >> >> > anywhere in the application. Please help me in resolving this
> problem.
> >> >> >
> >> >> > Thanks,
> >> >> > Anil Gupta
> >> >> >
> >> >>
> >> >
> >> >
> >> >
> >> > --
> >> > Thanks & Regards,
> >> > Anil Gupta
> >>
> >
> >
> >
> > --
> > Thanks & Regards,
> > Anil Gupta
>
+
Harsh J 2012-07-27, 21:23
+
anil gupta 2012-07-27, 22:05
+
Harsh J 2012-07-27, 22:36
+
anil gupta 2012-07-27, 23:22
+
Harsh J 2012-07-27, 23:39
+
anil gupta 2012-07-30, 23:03