Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> AvroStorage Issue - Possibly version related


Copy link to this message
-
Re: AvroStorage Issue - Possibly version related
Are the releases from the download page not compatible with 23.x? or 2.X

Says they are -
http://pig.apache.org/releases.html#1+April%2C+2013%3A+release+0.11.1+available

In any case I tried it with .23.9 and received a different error:

2013-09-19 15:13:56,044 [main] WARN
org.apache.pig.backend.hadoop20.PigJobControl - falling back to default
JobControl (not using hadoop 0.20 ?)
java.lang.NoSuchFieldException: runnerState
    at java.lang.Class.getDeclaredField(Class.java:1938)

On Thu, Sep 19, 2013 at 2:24 PM, Mark Wagner <[EMAIL PROTECTED]>wrote:

> It sounds like you're using a version of Pig that wasn't compiled for
> Hadoop 2.x/.23. Try recompiling with 'ant clean jar
> -Dhadoopversion=23'.
>
> -Mark
>
> On Thu, Sep 19, 2013 at 9:23 AM, j.barrett Strausser
> <[EMAIL PROTECTED]> wrote:
> > Running
> >
> > Hadoop-2.1.0-Beta
> > Pig-0.11.1
> > Hive-0.11.1
> >
> > 1. Created Avro backed table in Hive.
> > 2. Loaded the table in Pig - records = Load '/path' USING
> > org.apache.pig.piggybank.storage.avro.AvroStorage();
> > 3. Can successfully describe the relation.
> >
> > I registered the following on pig start :
> > REGISTER piggybank.jar
> > REGISTER avro-*.jar
> > REGISTER jackson-core-asl-1.8.8.jar
> > REGISTER jackson-mapper-asl-1.8.8.jar
> > REGISTER json-simple-1.1.jar
> > REGISTER snappy-java-1.0.3.2.jar
> >
> > The avro tools are at 1.7.5
> >
> >
> >
> >
> >
> > *
> > *
> > *Running Dump Produces the following*:
> >
> > 2013-09-19 12:08:21,639 [JobControl] ERROR
> > org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl - Error while
> trying
> > to run jobs.
> > java.lang.IncompatibleClassChangeError: Found interface
> > org.apache.hadoop.mapreduce.JobContext, but class was expected
> >     at
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.setupUdfEnvAndStores(PigOutputFormat.java:225)
> >     at
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.checkOutputSpecs(PigOutputFormat.java:186)
> >     at
> >
> org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:441)
> >     at
> >
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:340)
> >
> >
> >
> > 2013-09-19 12:08:21,651 [main] ERROR
> > org.apache.pig.tools.pigstats.SimplePigStats - ERROR 2997: Unable to
> > recreate exception from backend error: Unexpected System Error Occured:
> > java.lang.IncompatibleClassChangeError: Found interface
> > org.apache.hadoop.mapreduce.JobContext, but class was expected
> >     at
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.setupUdfEnvAndStores(PigOutputFormat.java:225)
> >     at
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.checkOutputSpecs(PigOutputFormat.java:186)
> >     at
> >
> org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:441)
> >     at
> >
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:340)
> >
> > *Running illustrate :*
> >
> > Pig Stack Trace
> > ---------------
> > ERROR 1070: Could not resolve
> > org.apache.pig.piggybank.storage.avro.AvroStorage using imports: [,
> > org.apache.pig.builtin., org.apache.pig.impl.builtin.]
> >
> > Pig Stack Trace
> > ---------------
> > ERROR 2998: Unhandled internal error.
> >
> org.apache.hadoop.mapreduce.Mapper$Context.<init>(Lorg/apache/hadoop/mapreduce/Mapper;Lorg/apache/hadoop/conf/Configuration;Lorg/apache/hadoop/mapreduce/TaskAttemptID;L$
> >
> > java.lang.NoSuchMethodError:
> >
> org.apache.hadoop.mapreduce.Mapper$Context.<init>(Lorg/apache/hadoop/mapreduce/Mapper;Lorg/apache/hadoop/conf/Configuration;Lorg/apache/hadoop/mapreduce/TaskAttemptID;Lorg/apach$
> >
> >
> >
> > Any thougths?
> >
> > --
> >
> >
> > https://github.com/bearrito
> > @deepbearrito
>

--
https://github.com/bearrito
@deepbearrito
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB