Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> temporary file locations for YARN applications


Copy link to this message
-
Re: temporary file locations for YARN applications
Every container gets its own local work directory (You can use the
relative ./) thats auto-cleaned up at the end of the container's life.
This is the best place to store the temporary files. This is not
something you need custom configuration for.

Do the files need to be on a distributed FS or a local one?

On Sun, Oct 20, 2013 at 8:54 PM, John Lilley <[EMAIL PROTECTED]> wrote:
> We have a pure YARN application (no MapReduce) that has need to store a
> significant amount of temporary data.  How can we know the best location for
> these files?  How can we ensure that our YARN tasks have write access to
> these locations?  Is this something that must be configured outside of YARN?
>
> Thanks,
>
> John
>
>
>
>

--
Harsh J
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB