Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Simple AvroStorage LOAD and STORE with Avro 1.6.0


Copy link to this message
-
Re: Simple AvroStorage LOAD and STORE with Avro 1.6.0
Andrew,

The source of the problem may be AvroStorage in piggybank.  Could you
please include the entire stack trace?

stan

On Mon, Jan 9, 2012 at 4:15 AM, Andrew Kenworthy <[EMAIL PROTECTED]> wrote:
> Hallo,
>
> When I run a simple pig script to LOAD and STORE avro data, I get:-
>
> java.lang.ClassCastException: org.apache.pig.data.BinSedesTuple cannot be cast to org.apache.avro.generic.IndexedRecord
>
>
> Script:
>
> REGISTER /tmp/avro-1.6.0.jar;
> --REGISTER /tmp/avro-1.5.4.jar
> --REGISTER /tmp/avro-1.4.1.jar;
>
> REGISTER /tmp/piggybank-0.9.1.jar;
> REGISTER /tmp/json-simple-1.1.jar;
> REGISTER /tmp/jackson-core-asl-1.8.4.jar;
> REGISTER /tmp/jackson-mapper-asl-1.8.4.jar;
>
> avroData=LOAD '$DATA_INPUTDIR' USING org.apache.pig.piggybank.storage.avro.AvroStorage();
>
> dataSubset = FOREACH avroData GENERATE myField1, myField2;
> describe dataSubset;
> -----------------------------------------------
> -- shows:
> -- dataSubset : {myField1: int,myField2: int}
> -----------------------------------------------
> STORE dataSubset INTO '$OUTPUTDIR' USING org.apache.pig.piggybank.storage.avro.AvroStorage();
>
> If I use the 1.5.4 jar I get the same error, but the script works with the 1.4.1 version. If I just write one field, then it works with 1.6.0.
>
> I see there's been a related issue fixed here:
>
> https://issues.apache.org/jira/browse/PIG-2202
> https://issues.apache.org/jira/browse/PIG-2195
>
> Can anyone confirm that this or similar works with avro 1.6.0, and/or point me in the right direction concering where the problem may lie?
>
> Many thanks,
>
> Andrew
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB