Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - YARN Pi example job stuck at 0%(No MR tasks are started by ResourceManager)


Copy link to this message
-
Re: YARN Pi example job stuck at 0%(No MR tasks are started by ResourceManager)
anil gupta 2012-07-31, 18:26
Hi Harsh and Others,

I was able to run the job when I login as user "hdfs". However, it fails if
i run it as "root". I was suspecting this as a problem before also and it
came out to be true.

Thanks,
Anil gupta

On Mon, Jul 30, 2012 at 9:21 PM, abhiTowson cal
<[EMAIL PROTECTED]>wrote:

> Hi anil,
>
> Was trying several things.I didn't hadoop-env.sh, so i created it.
>
> Regards
> Abhishek
>
> On Mon, Jul 30, 2012 at 11:51 PM, anil gupta <[EMAIL PROTECTED]>
> wrote:
> > That's pretty interesting!! From where you figured out that you need to
> add
> > that property? Just trying to understand how adding that property fixed
> the
> > issue.
> >
> > On Mon, Jul 30, 2012 at 8:12 PM, abhiTowson cal
> > <[EMAIL PROTECTED]>wrote:
> >
> >> Hi anil,
> >>
> >> Adding property resolved issue for me, and i also made this change
> >>
> >> vim hadoop-env.sh
> >>
> >> export JAVA_HOME=/usr/lib/java-1.6.0/jdk1.6.0_33
> >> if [ "$JAVA_HOME" != "" ]; then
> >>   #echo "run java in $JAVA_HOME"
> >>   JAVA_HOME=$JAVA_HOME
> >> fi
> >>
> >> if [ "$JAVA_HOME" = "" ]; then
> >>   echo "Error: JAVA_HOME is not set."
> >>   exit 1
> >> fi
> >>
> >> JAVA=$JAVA_HOME/bin/java
> >> JAVA_HEAP_MAX=-Xmx1000m
> >>
> >> Regards
> >> Abhishek
> >>
> >>
> >> On Mon, Jul 30, 2012 at 10:47 PM, anil gupta <[EMAIL PROTECTED]>
> >> wrote:
> >> > Hi Abhishek,
> >> >
> >> > Did you mean that adding yarn.resourcemanager.resource-tracker.address
> >> > along with yarn.log-aggregation-enable in my configuration will
> resolve
> >> the
> >> > problem in which map-reduce job fails at 0% with the following error:
> In
> >> > the web page of
> >> >
> >>
> http://data-node:8042/node/containerlogs/container_1343687008058_0003_01_000001/rootthe
> >> > page says:
> >> > Failed redirect for container_1343687008058_0003_01_000001  Failed
> while
> >> > trying to construct the redirect url to the log server. Log Server url
> >> may
> >> > not be configured. Unknown container. Container either has not
> started or
> >> > has already completed or doesn't belong to this node at all.
> >> > Please let me know.
> >> >
> >> > Thanks,
> >> > Anil Gupta
> >> >
> >> > On Mon, Jul 30, 2012 at 7:30 PM, abhiTowson cal
> >> > <[EMAIL PROTECTED]>wrote:
> >> >
> >> >> hi anil,
> >> >>
> >> >> Adding these help me resolve the issue for me
> >> >> yarn.resourcemanager.resource-tracker.address
> >> >>
> >> >> Regards
> >> >> Abhishek
> >> >>
> >> >> On Mon, Jul 30, 2012 at 7:56 PM, anil gupta <[EMAIL PROTECTED]>
> >> wrote:
> >> >> > Hi Rahul,
> >> >> >
> >> >> > Thanks for your response. I can certainly enable the
> >> >> > yarn.log-aggregation-enable to true. But after enabling this what
> >> manual
> >> >> > steps i will have to take to run jobs. Could you please elaborate.
> >> >> >
> >> >> > Thanks,
> >> >> > Anil
> >> >> >
> >> >> > On Mon, Jul 30, 2012 at 4:26 PM, Rahul Jain <[EMAIL PROTECTED]>
> wrote:
> >> >> >
> >> >> >> The inability to look at map-reduce logs for failed logs is due to
> >> >> number
> >> >> >> of open issues in yarn; see my recent comment here:
> >> >> >>
> >> >> >>
> >> >>
> >>
> https://issues.apache.org/jira/browse/MAPREDUCE-4428?focusedCommentId=13412995&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13412995
> >> >> >>
> >> >> >> You can workaround this by enabling log aggregation and manually
> >> copying
> >> >> >> job logs from HDFS log location. Of course that is a painful way
> till
> >> >> the
> >> >> >> yarn log collection and history bugs are resolved in an upcoming
> >> >> release.
> >> >> >>
> >> >> >> -Rahul
> >> >> >>
> >> >> >>
> >> >> >> > 12/07/27 09:38:27 INFO mapred.ResourceMgrDelegate: Submitted
> >> >> application
> >> >> >> > application_1343365114818_0002 to ResourceManager at ihub-an-l1/
> >> >> >> > 172.31.192.151:8040
> >> >> >> > 12/07/27 09:38:27 INFO mapreduce.Job: The url to track the job:
> >> >> >> > http://ihub-an-l1:9999/proxy/application_1343365114818_0002/
> >> >> >> > 12/07/27 09:38:27 INFO mapreduce.Job: Running job:

Thanks & Regards,
Anil Gupta