Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> Symbolic links available in 1.0.3?


Copy link to this message
-
Symbolic links available in 1.0.3?
Is it possible to use symbolic links in 1.0.3?

 

If yes: can I use symbolic links to create a single, final directory
structure of files from many locations; then use DistCp/S3DistCp to copy
that final directory structure to another filesystem such as S3?

 

Usecase:

I currently launch 4 S3DistCp jobs concurrently (a royal pain in the a** to
do) and copy files from 4 different locations to various subfolders of a
single directory on S3 (the final result directory).

 

I'd love to create 1 S3DistCp job to do all of that work, but
S3DistCp/DistCp don't support specifying multiple source/destinations.