Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - Pig Output


Copy link to this message
-
Re: Pig Output
Russell Jurney 2011-12-05, 21:39
hadoop dfs cat /my/path/* > single_file

Russell Jurney
twitter.com/rjurney
[EMAIL PROTECTED]
datasyndrome.com

On Dec 5, 2011, at 12:30 PM, Aaron Griffith <[EMAIL PROTECTED]> wrote:

> Using PigStorage() my pig script output gets put into partial files on the hadoop
> file system.
>
> When I use the copyToLocal fuction from Hadoop it creates a local directory with
> all the partial files.
>
> Is there a way to copy the partial files from hadoop into a single local file?
>
> Thanks
>