Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - Support for Hadoop 2.2


Copy link to this message
-
Re: Support for Hadoop 2.2
Prashant Kommireddi 2013-10-25, 00:33
Yes it does. You need to recompile Pig for hadoop 2

ant clean jar-withouthadoop -Dhadoopversion=23
On Thu, Oct 24, 2013 at 5:37 AM, Claudio Romo Otto <
[EMAIL PROTECTED]> wrote:

> Does Pig support Hadoop 2.2? When I try Pig 0.12 and Hadoop 2.2 I get an
> error even with simple operations like
>
> data = LOAD 'cql://keyspace1/testcf?' USING CqlStorage();
> dump data;
>
> I only got a warning first and then and exception:
>
> 2013-10-24 09:35:19,300 [main] WARN org.apache.pig.backend.**hadoop20.PigJobControl
> - falling back to default JobControl (not using hadoop 0.20 ?)
> java.lang.**NoSuchFieldException: runnerState
>     at java.lang.Class.**getDeclaredField(Class.java:**1938)
>     at org.apache.pig.backend.**hadoop20.PigJobControl.<**
> clinit>(PigJobControl.java:51)
>     at org.apache.pig.backend.hadoop.**executionengine.shims.**
> HadoopShims.newJobControl(**HadoopShims.java:97)
>     at org.apache.pig.backend.hadoop.**executionengine.**mapReduceLayer.**
> JobControlCompiler.compile(**JobControlCompiler.java:285)
>     at org.apache.pig.backend.hadoop.**executionengine.**mapReduceLayer.**
> MapReduceLauncher.launchPig(**MapReduceLauncher.java:177)
>     at org.apache.pig.PigServer.**launchPlan(PigServer.java:**1264)
>     at org.apache.pig.PigServer.**executeCompiledLogicalPlan(**
> PigServer.java:1249)
>     at org.apache.pig.PigServer.**storeEx(PigServer.java:931)
>     at org.apache.pig.PigServer.**store(PigServer.java:898)
>     at org.apache.pig.PigServer.**openIterator(PigServer.java:**811)
>     at org.apache.pig.tools.grunt.**GruntParser.processDump(**
> GruntParser.java:696)
>     at org.apache.pig.tools.**pigscript.parser.**PigScriptParser.parse(**
> PigScriptParser.java:320)
>     at org.apache.pig.tools.grunt.**GruntParser.parseStopOnError(**
> GruntParser.java:194)
>     at org.apache.pig.tools.grunt.**GruntParser.parseStopOnError(**
> GruntParser.java:170)
>     at org.apache.pig.tools.grunt.**Grunt.run(Grunt.java:69)
>     at org.apache.pig.Main.run(Main.**java:538)
>     at org.apache.pig.Main.main(Main.**java:157)
>     at sun.reflect.**NativeMethodAccessorImpl.**invoke0(Native Method)
>     at sun.reflect.**NativeMethodAccessorImpl.**invoke(**
> NativeMethodAccessorImpl.java:**57)
>     at sun.reflect.**DelegatingMethodAccessorImpl.**invoke(**
> DelegatingMethodAccessorImpl.**java:43)
>     at java.lang.reflect.Method.**invoke(Method.java:606)
>     at org.apache.hadoop.util.RunJar.**main(RunJar.java:212)
>
> ------------------------------**--------
>
> Backend error message during job submission
> ------------------------------**-------------
> Unexpected System Error Occured: java.lang.**IncompatibleClassChangeError:
> Found interface org.apache.hadoop.mapreduce.**JobContext, but class was
> expected
>         at org.apache.pig.backend.hadoop.**executionengine.**
> mapReduceLayer.**PigOutputFormat.**setupUdfEnvAndStores(**
> PigOutputFormat.java:225)
>         at org.apache.pig.backend.hadoop.**executionengine.**
> mapReduceLayer.**PigOutputFormat.**checkOutputSpecs(**
> PigOutputFormat.java:186)
>         at org.apache.hadoop.mapreduce.**JobSubmitter.checkSpecs(**
> JobSubmitter.java:456)
>         at org.apache.hadoop.mapreduce.**JobSubmitter.**submitJobInternal(
> **JobSubmitter.java:342)
>         at org.apache.hadoop.mapreduce.**Job$10.run(Job.java:1268)
>         at org.apache.hadoop.mapreduce.**Job$10.run(Job.java:1265)
>         at java.security.**AccessController.doPrivileged(**Native Method)
>         at javax.security.auth.Subject.**doAs(Subject.java:415)
>         at org.apache.hadoop.security.**UserGroupInformation.doAs(**
> UserGroupInformation.java:**1491)
>         at org.apache.hadoop.mapreduce.**Job.submit(Job.java:1265)
>         at org.apache.hadoop.mapreduce.**lib.jobcontrol.ControlledJob.**
> submit(ControlledJob.java:335)
>         at org.apache.hadoop.mapreduce.**lib.jobcontrol.JobControl.run(**
> JobControl.java:240)
>         at org.apache.pig.backend.**hadoop20.PigJobControl.run(**