Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - Write a file to local disks on all nodes of a YARN cluster.


Copy link to this message
-
Write a file to local disks on all nodes of a YARN cluster.
Jay Vyas 2013-12-08, 21:25
I want to put a file on all nodes of my cluster, that is locally readable
(not in HDFS).

Assuming that i cant gaurantee a FUSE mount or NFS or anything of the SORT
on my cluster, is there a poor man's way to do this in yarn?  something
like

for each node n in cluster:
    n.copToLocal("a","/tmp/a");

So that afterwards, all nodes in the cluster have a file "a" in /tmp.

--
Jay Vyas
http://jayunit100.blogspot.com