Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - Compressing output using block compression


Copy link to this message
-
Re: Compressing output using block compression
Mohit Anchlia 2012-03-29, 23:07
Thanks! When I store output how can I tell pig to compress it in LZO format?

On Thu, Mar 29, 2012 at 4:02 PM, Dmitriy Ryaboy <[EMAIL PROTECTED]> wrote:

> You might find the elephant-bird project helpful for reading and
> creating LZO files, in raw hadoop or using Pig.
> (disclaimer: I'm a committer on elephant-bird)
>
> D
>
> On Wed, Mar 28, 2012 at 9:49 AM, Prashant Kommireddi
> <[EMAIL PROTECTED]> wrote:
> > Pig support LZO for splittable compression.
> >
> > Thanks,
> > Prashant
> >
> > On Mar 28, 2012, at 9:45 AM, Mohit Anchlia <[EMAIL PROTECTED]>
> wrote:
> >
> >> We currently have 100s of GB of uncompressed data which we would like to
> >> zip using some compression that is block compression so that we can use
> >> multiple input splits. Does pig support any such compression?
>