Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro, mail # user - can avro files on hdfs be read using pig


Copy link to this message
-
Re: can avro files on hdfs be read using pig
Sean Busbey 2013-09-29, 01:13
+user@pig

On Wed, Sep 25, 2013 at 9:33 AM, Anup <[EMAIL PROTECTED]> wrote:
> > On Sep 24, 2013, at 11:15 PM, Phani <[EMAIL PROTECTED]> wrote:
> >
> > wanted to know if avro files can be read using pig from hdfs.
> >
> > Thanks
> >
> > --
> > phani
>
> Yes. Use AvroStorage().
>
> Sent from my Turing Machine.
>

With slightly more detail:

AvroStorage will let you load and store arbitrary Avro:

https://cwiki.apache.org/confluence/display/PIG/AvroStorage

It is included in the contrib packages known as PiggyBank:

https://cwiki.apache.org/confluence/display/PIG/PiggyBank

Please note that the PiggyBank is distributed as source-only via SVN. The
page above explains how to build it.

If you use a repackaging of Pig provided by a commercial vendor, they will
likely provide a binary distribution and instructions specific to their
distribution for using AvroStorage.
--
Sean