Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Re: What makes a map to fail?


Copy link to this message
-
Re: What makes a map to fail?
<!--yarn-site.xml-->
 <property>
      <description>Whether to enable log aggregation</description>
          <name>yarn.log-aggregation-enable</name>
              <value>true</value>
 </property>

yarn.nodemanager.remote-app-log-dir
Specifies the directory where logs are aggregated.

Create the /var/log/hadoop-yarn directory and set ownership:
sudo -u hdfs hadoop fs -mkdir /var/log/hadoop-yarn
sudo -u hdfs hadoop fs -chown yarn:mapred /var/log/hadoop-yarn

You need to create this directory because it is the parent of
/var/log/hadoop-yarn/apps which is explicitly configured in the
yarn-site.xml.
FYI:
http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH4-Installation-Guide/cdh4ig_topic_11_4.html

2014/1/7 Saeed Adel Mehraban <[EMAIL PROTECTED]>

> When I click on individual maps logs, it says "Aggregation is not enabled.
> Try the nodemanager at slave1-machine:60933"
> How could I enable aggregation?
>
>
> On Sun, Jan 5, 2014 at 1:21 PM, Harsh J <[EMAIL PROTECTED]> wrote:
>
>> Every failed task typically carries a diagnostic message and a set of
>> logs for you to investigate what caused it to fail. Try visiting the
>> task's logs on the JT UI by clicking through individual failed
>> attempts to find the reason of its failure.
>>
>> On Sun, Jan 5, 2014 at 11:03 PM, Saeed Adel Mehraban <[EMAIL PROTECTED]>
>> wrote:
>> > Hi all,
>> > My task jobs are failing due to many failed maps. I want to know what
>> makes
>> > a map to fail? Is it something like exceptions or what?
>>
>>
>>
>> --
>> Harsh J
>>
>
>
--
Bing Jiang
weibo: http://weibo.com/jiangbinglover
BLOG: www.binospace.com
BLOG: http://blog.sina.com.cn/jiangbinglover
Focus on distributed computing, HDFS/HBase
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB