Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> YARN Pi example job stuck at 0%(No MR tasks are started by ResourceManager)


+
anil gupta 2012-07-27, 18:23
+
Rahul Jain 2012-07-30, 23:26
+
anil gupta 2012-07-30, 23:56
+
abhiTowson cal 2012-07-31, 02:30
+
anil gupta 2012-07-31, 02:47
+
abhiTowson cal 2012-07-31, 03:12
+
anil gupta 2012-07-31, 03:51
+
abhiTowson cal 2012-07-31, 04:21
+
anil gupta 2012-07-31, 18:26
Copy link to this message
-
Re: YARN Pi example job stuck at 0%(No MR tasks are started by ResourceManager)
Hi Harsh,

I created the following JIRA for NodeManager not reporting appropriate
errors when NM Memory < AM Memory:
 MAPREDUCE-4508 - YARN needs to properly check the NM,AM memory properties
in yarn-site.xml and mapred.xml and report errors
accordingly.<https://issues.apache.org/jira/browse/MAPREDUCE-4508>

Please let me know if anything else if required for the JIRA.

Thanks,
Anil Gupta

On Tue, Jul 31, 2012 at 11:26 AM, anil gupta <[EMAIL PROTECTED]> wrote:

> Hi Harsh and Others,
>
> I was able to run the job when I login as user "hdfs". However, it fails
> if i run it as "root". I was suspecting this as a problem before also and
> it came out to be true.
>
> Thanks,
> Anil gupta
>
>
> On Mon, Jul 30, 2012 at 9:21 PM, abhiTowson cal <[EMAIL PROTECTED]
> > wrote:
>
>> Hi anil,
>>
>> Was trying several things.I didn't hadoop-env.sh, so i created it.
>>
>> Regards
>> Abhishek
>>
>> On Mon, Jul 30, 2012 at 11:51 PM, anil gupta <[EMAIL PROTECTED]>
>> wrote:
>> > That's pretty interesting!! From where you figured out that you need to
>> add
>> > that property? Just trying to understand how adding that property fixed
>> the
>> > issue.
>> >
>> > On Mon, Jul 30, 2012 at 8:12 PM, abhiTowson cal
>> > <[EMAIL PROTECTED]>wrote:
>> >
>> >> Hi anil,
>> >>
>> >> Adding property resolved issue for me, and i also made this change
>> >>
>> >> vim hadoop-env.sh
>> >>
>> >> export JAVA_HOME=/usr/lib/java-1.6.0/jdk1.6.0_33
>> >> if [ "$JAVA_HOME" != "" ]; then
>> >>   #echo "run java in $JAVA_HOME"
>> >>   JAVA_HOME=$JAVA_HOME
>> >> fi
>> >>
>> >> if [ "$JAVA_HOME" = "" ]; then
>> >>   echo "Error: JAVA_HOME is not set."
>> >>   exit 1
>> >> fi
>> >>
>> >> JAVA=$JAVA_HOME/bin/java
>> >> JAVA_HEAP_MAX=-Xmx1000m
>> >>
>> >> Regards
>> >> Abhishek
>> >>
>> >>
>> >> On Mon, Jul 30, 2012 at 10:47 PM, anil gupta <[EMAIL PROTECTED]>
>> >> wrote:
>> >> > Hi Abhishek,
>> >> >
>> >> > Did you mean that adding
>> yarn.resourcemanager.resource-tracker.address
>> >> > along with yarn.log-aggregation-enable in my configuration will
>> resolve
>> >> the
>> >> > problem in which map-reduce job fails at 0% with the following
>> error: In
>> >> > the web page of
>> >> >
>> >>
>> http://data-node:8042/node/containerlogs/container_1343687008058_0003_01_000001/rootthe
>> >> > page says:
>> >> > Failed redirect for container_1343687008058_0003_01_000001  Failed
>> while
>> >> > trying to construct the redirect url to the log server. Log Server
>> url
>> >> may
>> >> > not be configured. Unknown container. Container either has not
>> started or
>> >> > has already completed or doesn't belong to this node at all.
>> >> > Please let me know.
>> >> >
>> >> > Thanks,
>> >> > Anil Gupta
>> >> >
>> >> > On Mon, Jul 30, 2012 at 7:30 PM, abhiTowson cal
>> >> > <[EMAIL PROTECTED]>wrote:
>> >> >
>> >> >> hi anil,
>> >> >>
>> >> >> Adding these help me resolve the issue for me
>> >> >> yarn.resourcemanager.resource-tracker.address
>> >> >>
>> >> >> Regards
>> >> >> Abhishek
>> >> >>
>> >> >> On Mon, Jul 30, 2012 at 7:56 PM, anil gupta <[EMAIL PROTECTED]>
>> >> wrote:
>> >> >> > Hi Rahul,
>> >> >> >
>> >> >> > Thanks for your response. I can certainly enable the
>> >> >> > yarn.log-aggregation-enable to true. But after enabling this what
>> >> manual
>> >> >> > steps i will have to take to run jobs. Could you please elaborate.
>> >> >> >
>> >> >> > Thanks,
>> >> >> > Anil
>> >> >> >
>> >> >> > On Mon, Jul 30, 2012 at 4:26 PM, Rahul Jain <[EMAIL PROTECTED]>
>> wrote:
>> >> >> >
>> >> >> >> The inability to look at map-reduce logs for failed logs is due
>> to
>> >> >> number
>> >> >> >> of open issues in yarn; see my recent comment here:
>> >> >> >>
>> >> >> >>
>> >> >>
>> >>
>> https://issues.apache.org/jira/browse/MAPREDUCE-4428?focusedCommentId=13412995&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13412995
>> >> >> >>
>> >> >> >> You can workaround this by enabling log aggregation and manually
>> >> copying

Thanks & Regards,
Anil Gupta
+
Rahul Jain 2012-07-31, 03:44
+
Harsh J 2012-07-27, 21:23
+
anil gupta 2012-07-27, 22:05
+
Harsh J 2012-07-27, 22:36
+
anil gupta 2012-07-27, 23:22
+
Harsh J 2012-07-27, 23:39
+
anil gupta 2012-07-30, 23:03
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB