Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig, mail # user - LZO & Pig (Elephantbird?)


+
Evert Lammerts 2011-01-12, 20:10
+
Tyler Coffin 2011-01-12, 20:23
+
Dmitriy Ryaboy 2011-01-12, 22:44
+
Dmitriy Ryaboy 2011-01-12, 22:51
+
Evert Lammerts 2011-01-13, 15:43
+
Dmitriy Ryaboy 2011-01-13, 19:26
+
Dmitriy Lyubimov 2011-01-20, 20:16
+
Dmitriy Ryaboy 2011-01-20, 20:41
+
Dmitriy Lyubimov 2011-01-20, 20:44
Copy link to this message
-
Re: LZO & Pig (Elephantbird?)
Evert Lammerts 2011-01-13, 20:18
Alright, thanks! I'm going to give that a try and will let you know how it goes.

Cheers, Evert

----- Reply message -----
From: "Dmitriy Ryaboy" <[EMAIL PROTECTED]>
Date: Thu, Jan 13, 2011 8:27 pm
Subject: LZO & Pig (Elephantbird?)
To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>

Depends on your definition of ready. I haven't put it into production and
there's a bit of clean-up left, but as far as I know there are no critical
bugs; just some interface issues which are shared between the current master
and the version for Pig 0.8.

Pig 0.8 should work with Hadoop 0.20.1; I've been working with it under CDH2
(which is a patched 0.20.1).

-Dmitriy

On Thu, Jan 13, 2011 at 7:43 AM, Evert Lammerts <[EMAIL PROTECTED]>wrote:

> > Are you trying to load protobufs or thrift files, or do you just want
> > Lzo
> > support?
>
> Protobufs would be nice, but Elephant Bird is not ready yet for Pig 0.7 /
> 0.8, right?
>
> > If you just want plain text lzo loading, the loaders in the
> > pig-08
> > branch totally work.
>
> Does Pig 0.8 work with Hadoop 0.20.1 as well?
>
> Thanks for the support!
> Evert
>
> >
> > Let me know if you have any issues.
> >
> > D
> >
> > On Wed, Jan 12, 2011 at 12:23 PM, Tyler Coffin <[EMAIL PROTECTED]> wrote:
> >
> > > There's a fork of elephant-bird where pig-8 support is being worked
> > on:
> > > https://github.com/dvryaboy/elephant-bird/tree/pig-08
> > >
> > > I haven't given it a shot yet.
> > >
> > > -----Original Message-----
> > > From: Evert Lammerts [mailto:[EMAIL PROTECTED]]
> > > Sent: January 12, 2011 15:10
> > > To: '[EMAIL PROTECTED]'
> > > Subject: LZO & Pig (Elephantbird?)
> > >
> > > Hello list,
> > >
> > > I've installed the LZO codecs (https://github.com/kevinweil/hadoop-
> > lzo)
> > > and
> > > now I'm looking into using LZO in Pig. Elephant Bird
> > > (https://github.com/kevinweil/elephant-bird) seems to provide some
> > nice
> > > prefab loaders, but it's requirements do not fit out Hadoop
> > installation
> > > (we're on CDH3b2 with Pig 0.7, EB cannot be used with anything >
> > 0.6). Also
> > > the need for Thrift 0.2 is unclear to me - Thrift is now at 0.5.
> > >
> > > Now I did find this project, http://code.google.com/p/hadoop-gpl-
> > packing/,
> > > saying EB can handle even Pig 0.8. This confuses me - can I or can I
> > not
> > > use
> > > Elephant Bird with Pig 0.7, or even upgrade to Pig 0.8?
> > >
> > > Since EB is probably not an option, does anybody have some pointers
> > on how
> > > to use LZO'ed files with Pig?
> > >
> > > Thanks!
> > >
> > > Evert Lammerts
> > >
> > > ---------------------------------------------------------------------
> > > This transmission (including any attachments) may contain
> > confidential
> > > information, privileged material (including material protected by the
> > > solicitor-client or other applicable privileges), or constitute non-
> > public
> > > information. Any use of this information by anyone other than the
> > intended
> > > recipient is prohibited. If you have received this transmission in
> > error,
> > > please immediately reply to the sender and delete this information
> > from your
> > > system. Use, dissemination, distribution, or reproduction of this
> > > transmission by unintended recipients is not authorized and may be
> > unlawful.
> > >
>
+
Gerrit van Vuuren 2011-01-17, 12:04