Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Support for Hadoop 2.2


Copy link to this message
-
Support for Hadoop 2.2
Does Pig support Hadoop 2.2? When I try Pig 0.12 and Hadoop 2.2 I get an
error even with simple operations like

data = LOAD 'cql://keyspace1/testcf?' USING CqlStorage();
dump data;

I only got a warning first and then and exception:

2013-10-24 09:35:19,300 [main] WARN
org.apache.pig.backend.hadoop20.PigJobControl - falling back to default
JobControl (not using hadoop 0.20 ?)
java.lang.NoSuchFieldException: runnerState
     at java.lang.Class.getDeclaredField(Class.java:1938)
     at
org.apache.pig.backend.hadoop20.PigJobControl.<clinit>(PigJobControl.java:51)
     at
org.apache.pig.backend.hadoop.executionengine.shims.HadoopShims.newJobControl(HadoopShims.java:97)
     at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:285)
     at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:177)
     at org.apache.pig.PigServer.launchPlan(PigServer.java:1264)
     at
org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1249)
     at org.apache.pig.PigServer.storeEx(PigServer.java:931)
     at org.apache.pig.PigServer.store(PigServer.java:898)
     at org.apache.pig.PigServer.openIterator(PigServer.java:811)
     at
org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:696)
     at
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:320)
     at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194)
     at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:170)
     at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
     at org.apache.pig.Main.run(Main.java:538)
     at org.apache.pig.Main.main(Main.java:157)
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
     at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
     at java.lang.reflect.Method.invoke(Method.java:606)
     at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

--------------------------------------

Backend error message during job submission
-------------------------------------------
Unexpected System Error Occured: java.lang.IncompatibleClassChangeError:
Found interface org.apache.hadoop.mapreduce.JobContext, but class was
expected
         at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.setupUdfEnvAndStores(PigOutputFormat.java:225)
         at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.checkOutputSpecs(PigOutputFormat.java:186)
         at
org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:456)
         at
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:342)
         at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1268)
         at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1265)
         at java.security.AccessController.doPrivileged(Native Method)
         at javax.security.auth.Subject.doAs(Subject.java:415)
         at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
         at org.apache.hadoop.mapreduce.Job.submit(Job.java:1265)
         at
org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:335)
         at
org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.run(JobControl.java:240)
         at
org.apache.pig.backend.hadoop20.PigJobControl.run(PigJobControl.java:121)
         at java.lang.Thread.run(Thread.java:724)
         at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:257)

Pig Stack Trace
---------------
ERROR 1066: Unable to open iterator for alias data

org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable
to open iterator for alias data
         at org.apache.pig.PigServer.openIterator(PigServer.java:836)
         at
org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:696)
         at
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:320)
         at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194)
         at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:170)
         at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
         at org.apache.pig.Main.run(Main.java:538)
         at org.apache.pig.Main.main(Main.java:157)
         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
         at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
         at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
         at java.lang.reflect.Method.invoke(Method.java:606)
         at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: java.io.IOException: Job terminated with anomalous status FAILED
         at org.apache.pig.PigServer.openIterator(PigServer.java:828)
         ... 12 more
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB