Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - Re: What makes a map to fail?


Copy link to this message
-
Re: What makes a map to fail?
Bing Jiang 2014-01-07, 08:35
<!--yarn-site.xml-->
 <property>
      <description>Whether to enable log aggregation</description>
          <name>yarn.log-aggregation-enable</name>
              <value>true</value>
 </property>

yarn.nodemanager.remote-app-log-dir
Specifies the directory where logs are aggregated.

Create the /var/log/hadoop-yarn directory and set ownership:
sudo -u hdfs hadoop fs -mkdir /var/log/hadoop-yarn
sudo -u hdfs hadoop fs -chown yarn:mapred /var/log/hadoop-yarn

You need to create this directory because it is the parent of
/var/log/hadoop-yarn/apps which is explicitly configured in the
yarn-site.xml.
FYI:
http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH4-Installation-Guide/cdh4ig_topic_11_4.html

2014/1/7 Saeed Adel Mehraban <[EMAIL PROTECTED]>

> When I click on individual maps logs, it says "Aggregation is not enabled.
> Try the nodemanager at slave1-machine:60933"
> How could I enable aggregation?
>
>
> On Sun, Jan 5, 2014 at 1:21 PM, Harsh J <[EMAIL PROTECTED]> wrote:
>
>> Every failed task typically carries a diagnostic message and a set of
>> logs for you to investigate what caused it to fail. Try visiting the
>> task's logs on the JT UI by clicking through individual failed
>> attempts to find the reason of its failure.
>>
>> On Sun, Jan 5, 2014 at 11:03 PM, Saeed Adel Mehraban <[EMAIL PROTECTED]>
>> wrote:
>> > Hi all,
>> > My task jobs are failing due to many failed maps. I want to know what
>> makes
>> > a map to fail? Is it something like exceptions or what?
>>
>>
>>
>> --
>> Harsh J
>>
>
>
--
Bing Jiang
weibo: http://weibo.com/jiangbinglover
BLOG: www.binospace.com
BLOG: http://blog.sina.com.cn/jiangbinglover
Focus on distributed computing, HDFS/HBase