Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - temporary folders for YARN tasks


Copy link to this message
-
temporary folders for YARN tasks
John Lilley 2013-07-01, 20:37
When a YARN app and its tasks wants to write temporary files, how does it know where to write the files?
I am assuming that each task has some temporary space available, and I hope it is available across multiple disk volumes for parallel performance.
Are those files cleaned up automatically after task exit?
If I want to give lifetime control of the files to an auxiliary service (along the lines of MR shuffle passing files to the aux service), how would I do that, and would that entail different file locations?
Thanks
John