Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - LZO & Pig (Elephantbird?)


Copy link to this message
-
RE: LZO & Pig (Elephantbird?)
Evert Lammerts 2011-01-13, 15:43
> Are you trying to load protobufs or thrift files, or do you just want
> Lzo
> support?

Protobufs would be nice, but Elephant Bird is not ready yet for Pig 0.7 /
0.8, right?

> If you just want plain text lzo loading, the loaders in the
> pig-08
> branch totally work.

Does Pig 0.8 work with Hadoop 0.20.1 as well?

Thanks for the support!
Evert

>
> Let me know if you have any issues.
>
> D
>
> On Wed, Jan 12, 2011 at 12:23 PM, Tyler Coffin <[EMAIL PROTECTED]> wrote:
>
> > There's a fork of elephant-bird where pig-8 support is being worked
> on:
> > https://github.com/dvryaboy/elephant-bird/tree/pig-08
> >
> > I haven't given it a shot yet.
> >
> > -----Original Message-----
> > From: Evert Lammerts [mailto:[EMAIL PROTECTED]]
> > Sent: January 12, 2011 15:10
> > To: '[EMAIL PROTECTED]'
> > Subject: LZO & Pig (Elephantbird?)
> >
> > Hello list,
> >
> > I've installed the LZO codecs (https://github.com/kevinweil/hadoop-
> lzo)
> > and
> > now I'm looking into using LZO in Pig. Elephant Bird
> > (https://github.com/kevinweil/elephant-bird) seems to provide some
> nice
> > prefab loaders, but it's requirements do not fit out Hadoop
> installation
> > (we're on CDH3b2 with Pig 0.7, EB cannot be used with anything >
> 0.6). Also
> > the need for Thrift 0.2 is unclear to me - Thrift is now at 0.5.
> >
> > Now I did find this project, http://code.google.com/p/hadoop-gpl-
> packing/,
> > saying EB can handle even Pig 0.8. This confuses me - can I or can I
> not
> > use
> > Elephant Bird with Pig 0.7, or even upgrade to Pig 0.8?
> >
> > Since EB is probably not an option, does anybody have some pointers
> on how
> > to use LZO'ed files with Pig?
> >
> > Thanks!
> >
> > Evert Lammerts
> >
> > ---------------------------------------------------------------------
> > This transmission (including any attachments) may contain
> confidential
> > information, privileged material (including material protected by the
> > solicitor-client or other applicable privileges), or constitute non-
> public
> > information. Any use of this information by anyone other than the
> intended
> > recipient is prohibited. If you have received this transmission in
> error,
> > please immediately reply to the sender and delete this information
> from your
> > system. Use, dissemination, distribution, or reproduction of this
> > transmission by unintended recipients is not authorized and may be
> unlawful.
> >