Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> Simple AvroStorage LOAD and STORE with Avro 1.6.0


+
Andrew Kenworthy 2012-01-09, 09:15
Copy link to this message
-
Re: Simple AvroStorage LOAD and STORE with Avro 1.6.0
Andrew,

The source of the problem may be AvroStorage in piggybank.  Could you
please include the entire stack trace?

stan

On Mon, Jan 9, 2012 at 4:15 AM, Andrew Kenworthy <[EMAIL PROTECTED]> wrote:
> Hallo,
>
> When I run a simple pig script to LOAD and STORE avro data, I get:-
>
> java.lang.ClassCastException: org.apache.pig.data.BinSedesTuple cannot be cast to org.apache.avro.generic.IndexedRecord
>
>
> Script:
>
> REGISTER /tmp/avro-1.6.0.jar;
> --REGISTER /tmp/avro-1.5.4.jar
> --REGISTER /tmp/avro-1.4.1.jar;
>
> REGISTER /tmp/piggybank-0.9.1.jar;
> REGISTER /tmp/json-simple-1.1.jar;
> REGISTER /tmp/jackson-core-asl-1.8.4.jar;
> REGISTER /tmp/jackson-mapper-asl-1.8.4.jar;
>
> avroData=LOAD '$DATA_INPUTDIR' USING org.apache.pig.piggybank.storage.avro.AvroStorage();
>
> dataSubset = FOREACH avroData GENERATE myField1, myField2;
> describe dataSubset;
> -----------------------------------------------
> -- shows:
> -- dataSubset : {myField1: int,myField2: int}
> -----------------------------------------------
> STORE dataSubset INTO '$OUTPUTDIR' USING org.apache.pig.piggybank.storage.avro.AvroStorage();
>
> If I use the 1.5.4 jar I get the same error, but the script works with the 1.4.1 version. If I just write one field, then it works with 1.6.0.
>
> I see there's been a related issue fixed here:
>
> https://issues.apache.org/jira/browse/PIG-2202
> https://issues.apache.org/jira/browse/PIG-2195
>
> Can anyone confirm that this or similar works with avro 1.6.0, and/or point me in the right direction concering where the problem may lie?
>
> Many thanks,
>
> Andrew
+
Andrew Kenworthy 2012-01-10, 13:03
+
Stan Rosenberg 2012-01-10, 16:36
+
Andrew Kenworthy 2012-01-11, 09:47
+
Scott Carey 2012-01-16, 21:16
+
Russell Jurney 2012-01-09, 20:47
+
Stan Rosenberg 2012-01-09, 20:52
+
Russell Jurney 2012-01-09, 21:21
+
Bill Graham 2012-01-10, 00:42