Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> YARN task default output


Copy link to this message
-
Re: YARN task default output
You can enable log-aggregation where-by your application logs will be uploaded to HDFS.

Overview here:
http://hortonworks.com/blog/apache-hadoop-yarn-nodemanager/

Configurations from http://hadoop.apache.org/docs/r2.0.4-alpha/hadoop-project-dist/hadoop-common/ClusterSetup.html which are relevant are:
yarn.log-aggregation-enable -> true
yarn.nodemanager.remote-app-log-dir -> /logs (directory on hdfs which will need to be created before you start your cluster)

hth,
Arun
On Jun 6, 2013, at 11:10 AM, John Lilley wrote:

> Do tasks that spawn from a YARN AM have a default working directory?  Does stdout/stderr get captured anywhere by default?
> I ask because I am setting up tests with the “distributed shell” AM and want to know if basic commands (e.g. ls) will send stdout/stderr somewhere I can see at the end.
> Thanks
> John
>  

--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB