Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> YARN task default output


Copy link to this message
-
Re: YARN task default output
You can enable log-aggregation where-by your application logs will be uploaded to HDFS.

Overview here:
http://hortonworks.com/blog/apache-hadoop-yarn-nodemanager/

Configurations from http://hadoop.apache.org/docs/r2.0.4-alpha/hadoop-project-dist/hadoop-common/ClusterSetup.html which are relevant are:
yarn.log-aggregation-enable -> true
yarn.nodemanager.remote-app-log-dir -> /logs (directory on hdfs which will need to be created before you start your cluster)

hth,
Arun
On Jun 6, 2013, at 11:10 AM, John Lilley wrote:

> Do tasks that spawn from a YARN AM have a default working directory?  Does stdout/stderr get captured anywhere by default?
> I ask because I am setting up tests with the “distributed shell” AM and want to know if basic commands (e.g. ls) will send stdout/stderr somewhere I can see at the end.
> Thanks
> John
>  

--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/