Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Write a file to local disks on all nodes of a YARN cluster.


Copy link to this message
-
Write a file to local disks on all nodes of a YARN cluster.
I want to put a file on all nodes of my cluster, that is locally readable
(not in HDFS).

Assuming that i cant gaurantee a FUSE mount or NFS or anything of the SORT
on my cluster, is there a poor man's way to do this in yarn?  something
like

for each node n in cluster:
    n.copToLocal("a","/tmp/a");

So that afterwards, all nodes in the cluster have a file "a" in /tmp.

--
Jay Vyas
http://jayunit100.blogspot.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB