Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig, mail # user - Support for Hadoop 2.2


Copy link to this message
-
Support for Hadoop 2.2
Claudio Romo Otto 2013-10-24, 12:37
Does Pig support Hadoop 2.2? When I try Pig 0.12 and Hadoop 2.2 I get an
error even with simple operations like

data = LOAD 'cql://keyspace1/testcf?' USING CqlStorage();
dump data;

I only got a warning first and then and exception:

2013-10-24 09:35:19,300 [main] WARN
org.apache.pig.backend.hadoop20.PigJobControl - falling back to default
JobControl (not using hadoop 0.20 ?)
java.lang.NoSuchFieldException: runnerState
     at java.lang.Class.getDeclaredField(Class.java:1938)
     at
org.apache.pig.backend.hadoop20.PigJobControl.<clinit>(PigJobControl.java:51)
     at
org.apache.pig.backend.hadoop.executionengine.shims.HadoopShims.newJobControl(HadoopShims.java:97)
     at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:285)
     at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:177)
     at org.apache.pig.PigServer.launchPlan(PigServer.java:1264)
     at
org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1249)
     at org.apache.pig.PigServer.storeEx(PigServer.java:931)
     at org.apache.pig.PigServer.store(PigServer.java:898)
     at org.apache.pig.PigServer.openIterator(PigServer.java:811)
     at
org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:696)
     at
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:320)
     at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194)
     at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:170)
     at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
     at org.apache.pig.Main.run(Main.java:538)
     at org.apache.pig.Main.main(Main.java:157)
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
     at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
     at java.lang.reflect.Method.invoke(Method.java:606)
     at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

--------------------------------------

Backend error message during job submission
-------------------------------------------
Unexpected System Error Occured: java.lang.IncompatibleClassChangeError:
Found interface org.apache.hadoop.mapreduce.JobContext, but class was
expected
         at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.setupUdfEnvAndStores(PigOutputFormat.java:225)
         at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.checkOutputSpecs(PigOutputFormat.java:186)
         at
org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:456)
         at
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:342)
         at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1268)
         at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1265)
         at java.security.AccessController.doPrivileged(Native Method)
         at javax.security.auth.Subject.doAs(Subject.java:415)
         at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
         at org.apache.hadoop.mapreduce.Job.submit(Job.java:1265)
         at
org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:335)
         at
org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.run(JobControl.java:240)
         at
org.apache.pig.backend.hadoop20.PigJobControl.run(PigJobControl.java:121)
         at java.lang.Thread.run(Thread.java:724)
         at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:257)

Pig Stack Trace
---------------
ERROR 1066: Unable to open iterator for alias data

org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable
to open iterator for alias data
         at org.apache.pig.PigServer.openIterator(PigServer.java:836)
         at
org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:696)
         at
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:320)
         at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194)
         at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:170)
         at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
         at org.apache.pig.Main.run(Main.java:538)
         at org.apache.pig.Main.main(Main.java:157)
         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
         at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
         at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
         at java.lang.reflect.Method.invoke(Method.java:606)
         at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: java.io.IOException: Job terminated with anomalous status FAILED
         at org.apache.pig.PigServer.openIterator(PigServer.java:828)
         ... 12 more
+
Prashant Kommireddi 2013-10-25, 00:33
+
Claudio Romo Otto 2013-10-25, 01:34
+
Jarek Jarcec Cecho 2013-10-25, 16:11
+
Claudio Romo Otto 2013-10-25, 19:10
+
Jarek Jarcec Cecho 2013-10-30, 21:03
+
Claudio Romo Otto 2013-10-30, 22:40
+
Juan Martin Pampliega 2013-11-26, 19:07
+
Claudio Romo Otto 2013-11-26, 20:32
+
Uttam Kumar 2013-11-27, 00:54