Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> AvroStorage Issue - Possibly version related


Copy link to this message
-
AvroStorage Issue - Possibly version related
Running

Hadoop-2.1.0-Beta
Pig-0.11.1
Hive-0.11.1

1. Created Avro backed table in Hive.
2. Loaded the table in Pig - records = Load '/path' USING
org.apache.pig.piggybank.storage.avro.AvroStorage();
3. Can successfully describe the relation.

I registered the following on pig start :
REGISTER piggybank.jar
REGISTER avro-*.jar
REGISTER jackson-core-asl-1.8.8.jar
REGISTER jackson-mapper-asl-1.8.8.jar
REGISTER json-simple-1.1.jar
REGISTER snappy-java-1.0.3.2.jar

The avro tools are at 1.7.5

*
*
*Running Dump Produces the following*:

2013-09-19 12:08:21,639 [JobControl] ERROR
org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl - Error while trying
to run jobs.
java.lang.IncompatibleClassChangeError: Found interface
org.apache.hadoop.mapreduce.JobContext, but class was expected
    at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.setupUdfEnvAndStores(PigOutputFormat.java:225)
    at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.checkOutputSpecs(PigOutputFormat.java:186)
    at
org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:441)
    at
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:340)

2013-09-19 12:08:21,651 [main] ERROR
org.apache.pig.tools.pigstats.SimplePigStats - ERROR 2997: Unable to
recreate exception from backend error: Unexpected System Error Occured:
java.lang.IncompatibleClassChangeError: Found interface
org.apache.hadoop.mapreduce.JobContext, but class was expected
    at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.setupUdfEnvAndStores(PigOutputFormat.java:225)
    at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.checkOutputSpecs(PigOutputFormat.java:186)
    at
org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:441)
    at
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:340)

*Running illustrate :*

Pig Stack Trace
---------------
ERROR 1070: Could not resolve
org.apache.pig.piggybank.storage.avro.AvroStorage using imports: [,
org.apache.pig.builtin., org.apache.pig.impl.builtin.]

Pig Stack Trace
---------------
ERROR 2998: Unhandled internal error.
org.apache.hadoop.mapreduce.Mapper$Context.<init>(Lorg/apache/hadoop/mapreduce/Mapper;Lorg/apache/hadoop/conf/Configuration;Lorg/apache/hadoop/mapreduce/TaskAttemptID;L$

java.lang.NoSuchMethodError:
org.apache.hadoop.mapreduce.Mapper$Context.<init>(Lorg/apache/hadoop/mapreduce/Mapper;Lorg/apache/hadoop/conf/Configuration;Lorg/apache/hadoop/mapreduce/TaskAttemptID;Lorg/apach$

Any thougths?

--
https://github.com/bearrito
@deepbearrito
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB