Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Pig Output


Copy link to this message
-
Re: Pig Output
Hi Aaron
         Instead of copyFromLocal use getmerge. It would do your job. The
syntax for CLI is
hadoop fs -getmerge <source dir in hdfs/pig output dir> <lfs destn
dir>/xyz.txt
Hope it helps!...

Regards
Bejoy.K.S

On Tue, Dec 6, 2011 at 1:57 AM, Aaron Griffith
<[EMAIL PROTECTED]>wrote:

> Using PigStorage() my pig script output gets put into partial files on the
> hadoop
> file system.
>
> When I use the copyToLocal fuction from Hadoop it creates a local
> directory with
> all the partial files.
>
> Is there a way to copy the partial files from hadoop into a single local
> file?
>
> Thanks
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB