Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Pig Output

Copy link to this message
Re: Pig Output
Hi Aaron
         Instead of copyFromLocal use getmerge. It would do your job. The
syntax for CLI is
hadoop fs -getmerge <source dir in hdfs/pig output dir> <lfs destn
Hope it helps!...


On Tue, Dec 6, 2011 at 1:57 AM, Aaron Griffith

> Using PigStorage() my pig script output gets put into partial files on the
> hadoop
> file system.
> When I use the copyToLocal fuction from Hadoop it creates a local
> directory with
> all the partial files.
> Is there a way to copy the partial files from hadoop into a single local
> file?
> Thanks