Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> YARN Pi example job stuck at 0%(No MR tasks are started by ResourceManager)


+
anil gupta 2012-07-27, 18:23
+
Rahul Jain 2012-07-30, 23:26
+
anil gupta 2012-07-30, 23:56
+
abhiTowson cal 2012-07-31, 02:30
+
anil gupta 2012-07-31, 02:47
+
abhiTowson cal 2012-07-31, 03:12
+
anil gupta 2012-07-31, 03:51
+
abhiTowson cal 2012-07-31, 04:21
+
anil gupta 2012-07-31, 18:26
Copy link to this message
-
Re: YARN Pi example job stuck at 0%(No MR tasks are started by ResourceManager)
Hi Harsh,

I created the following JIRA for NodeManager not reporting appropriate
errors when NM Memory < AM Memory:
 MAPREDUCE-4508 - YARN needs to properly check the NM,AM memory properties
in yarn-site.xml and mapred.xml and report errors
accordingly.<https://issues.apache.org/jira/browse/MAPREDUCE-4508>

Please let me know if anything else if required for the JIRA.

Thanks,
Anil Gupta

On Tue, Jul 31, 2012 at 11:26 AM, anil gupta <[EMAIL PROTECTED]> wrote:

> Hi Harsh and Others,
>
> I was able to run the job when I login as user "hdfs". However, it fails
> if i run it as "root". I was suspecting this as a problem before also and
> it came out to be true.
>
> Thanks,
> Anil gupta
>
>
> On Mon, Jul 30, 2012 at 9:21 PM, abhiTowson cal <[EMAIL PROTECTED]
> > wrote:
>
>> Hi anil,
>>
>> Was trying several things.I didn't hadoop-env.sh, so i created it.
>>
>> Regards
>> Abhishek
>>
>> On Mon, Jul 30, 2012 at 11:51 PM, anil gupta <[EMAIL PROTECTED]>
>> wrote:
>> > That's pretty interesting!! From where you figured out that you need to
>> add
>> > that property? Just trying to understand how adding that property fixed
>> the
>> > issue.
>> >
>> > On Mon, Jul 30, 2012 at 8:12 PM, abhiTowson cal
>> > <[EMAIL PROTECTED]>wrote:
>> >
>> >> Hi anil,
>> >>
>> >> Adding property resolved issue for me, and i also made this change
>> >>
>> >> vim hadoop-env.sh
>> >>
>> >> export JAVA_HOME=/usr/lib/java-1.6.0/jdk1.6.0_33
>> >> if [ "$JAVA_HOME" != "" ]; then
>> >>   #echo "run java in $JAVA_HOME"
>> >>   JAVA_HOME=$JAVA_HOME
>> >> fi
>> >>
>> >> if [ "$JAVA_HOME" = "" ]; then
>> >>   echo "Error: JAVA_HOME is not set."
>> >>   exit 1
>> >> fi
>> >>
>> >> JAVA=$JAVA_HOME/bin/java
>> >> JAVA_HEAP_MAX=-Xmx1000m
>> >>
>> >> Regards
>> >> Abhishek
>> >>
>> >>
>> >> On Mon, Jul 30, 2012 at 10:47 PM, anil gupta <[EMAIL PROTECTED]>
>> >> wrote:
>> >> > Hi Abhishek,
>> >> >
>> >> > Did you mean that adding
>> yarn.resourcemanager.resource-tracker.address
>> >> > along with yarn.log-aggregation-enable in my configuration will
>> resolve
>> >> the
>> >> > problem in which map-reduce job fails at 0% with the following
>> error: In
>> >> > the web page of
>> >> >
>> >>
>> http://data-node:8042/node/containerlogs/container_1343687008058_0003_01_000001/rootthe
>> >> > page says:
>> >> > Failed redirect for container_1343687008058_0003_01_000001  Failed
>> while
>> >> > trying to construct the redirect url to the log server. Log Server
>> url
>> >> may
>> >> > not be configured. Unknown container. Container either has not
>> started or
>> >> > has already completed or doesn't belong to this node at all.
>> >> > Please let me know.
>> >> >
>> >> > Thanks,
>> >> > Anil Gupta
>> >> >
>> >> > On Mon, Jul 30, 2012 at 7:30 PM, abhiTowson cal
>> >> > <[EMAIL PROTECTED]>wrote:
>> >> >
>> >> >> hi anil,
>> >> >>
>> >> >> Adding these help me resolve the issue for me
>> >> >> yarn.resourcemanager.resource-tracker.address
>> >> >>
>> >> >> Regards
>> >> >> Abhishek
>> >> >>
>> >> >> On Mon, Jul 30, 2012 at 7:56 PM, anil gupta <[EMAIL PROTECTED]>
>> >> wrote:
>> >> >> > Hi Rahul,
>> >> >> >
>> >> >> > Thanks for your response. I can certainly enable the
>> >> >> > yarn.log-aggregation-enable to true. But after enabling this what
>> >> manual
>> >> >> > steps i will have to take to run jobs. Could you please elaborate.
>> >> >> >
>> >> >> > Thanks,
>> >> >> > Anil
>> >> >> >
>> >> >> > On Mon, Jul 30, 2012 at 4:26 PM, Rahul Jain <[EMAIL PROTECTED]>
>> wrote:
>> >> >> >
>> >> >> >> The inability to look at map-reduce logs for failed logs is due
>> to
>> >> >> number
>> >> >> >> of open issues in yarn; see my recent comment here:
>> >> >> >>
>> >> >> >>
>> >> >>
>> >>
>> https://issues.apache.org/jira/browse/MAPREDUCE-4428?focusedCommentId=13412995&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13412995
>> >> >> >>
>> >> >> >> You can workaround this by enabling log aggregation and manually
>> >> copying

Thanks & Regards,
Anil Gupta
+
Rahul Jain 2012-07-31, 03:44
+
Harsh J 2012-07-27, 21:23
+
anil gupta 2012-07-27, 22:05
+
Harsh J 2012-07-27, 22:36
+
anil gupta 2012-07-27, 23:22
+
Harsh J 2012-07-27, 23:39
+
anil gupta 2012-07-30, 23:03