+
Nishant Neeraj 2012-10-26, 05:14
-
Re: Pig 0.10.0 and Hadoop 2.0.2: String JobControl.addJob(Job)Cheolsoo Park 2012-10-26, 06:12
Hi Nishant,
That's because the default pig jar file are compiled against hadoop-1.0.x,
but you're running it against hadoop-2.0.x. You need to re-build the pig
jar file.
Please do the following: (I am assuming that you're the installed hadoop
and pig-withouthadoop.jar.)
1) cd <pig home directory> (assuming that you have source code as well as
binaries.)
2) mv pig-0.10.0-withouthadoop.jar pig-0.10.0-withouthadoop.jar.bk
3) ant clean jar-withouthadoop -Dhadoopversion=23
You will see a new pig-0.10.0-withouthadoop.jar is built in the current
directory. Now you will no longer the error.
Thanks,
Cheolsoo
On Thu, Oct 25, 2012 at 10:14 PM, Nishant Neeraj <
[EMAIL PROTECTED]> wrote:
> I am new to Hadoop and Pig.
> I am trying to get Pig 0.10.0 [1] and Hadoop 2.0.2 [2] working together. I
> get
>
> java.lang.NoSuchMethodError:
>
> org.apache.hadoop.mapred.jobcontrol.JobControl.addJob(Lorg/apache/hadoop/mapred/jobcontrol/Job;)Ljava/lang/String;
>
> error, see complete stack trace in footnote [3]. How can I fix this?
>
> Here is what I am doing:
>
> grunt> cd hdfs:///data/
> grunt> cat test.data
> 2012-10-26 10:38:15,110 [main] WARN
> org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> 0 Wed Oct 24 14:35:21 IST 2012 name
> 1 Wed Oct 24 14:35:26 IST 2012 name
> 2 Wed Oct 24 14:35:31 IST 2012 name
> 3 Wed Oct 24 14:35:36 IST 2012 name
> 4 Wed Oct 24 14:35:41 IST 2012 name
> 5 Wed Oct 24 14:35:46 IST 2012 name
> 6 Wed Oct 24 14:35:51 IST 2012 name
> 7 Wed Oct 24 14:35:56 IST 2012 name
> 8 Wed Oct 24 14:36:01 IST 2012 name
> 9 Wed Oct 24 14:36:06 IST 2012 name
>
> grunt> data = LOAD 'test.data' ;
> 2012-10-26 10:38:29,389 [main] WARN org.apache.hadoop.conf.Configuration -
> mapred.jobtracker.maxtasks.per.job is deprecated. Instead, use
> mapreduce.jobtracker.maxtasks.perjob
> [ -- snip --]
>
> grunt> name = FILTER data BY $0 MATCHES '.*name.*' ;
> grunt> DUMP name
> [ -- snip -- ]
> 2012-10-26 10:39:23,299 [main] ERROR org.apache.pig.tools.grunt.Grunt -
> ERROR 2998: Unhandled internal error.
>
> org.apache.hadoop.mapred.jobcontrol.JobControl.addJob(Lorg/apache/hadoop/mapred/jobcontrol/Job;)Ljava/lang/String;
>
>
> ----
> [1]
> Apache Pig version 0.10.0 (r1328203)
> compiled Apr 19 2012, 22:54:12
>
> [2]
> Hadoop 2.0.2-alpha
> Subversion
>
> https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2.0.2-alpha/hadoop-common-project/hadoop-common-r
> 1392682
> Compiled by hortonmu on Tue Oct 2 00:44:10 UTC 2012
> From source with checksum efbdb59af73bfc103f1945d65dbf3071
>
> [3]
> Pig Stack Trace
> ---------------
> ERROR 2998: Unhandled internal error.
>
> org.apache.hadoop.mapred.jobcontrol.JobControl.addJob(Lorg/apache/hadoop/mapred/jobcontrol/Job;)Ljava/lang/String;
>
> java.lang.NoSuchMethodError:
>
> org.apache.hadoop.mapred.jobcontrol.JobControl.addJob(Lorg/apache/hadoop/mapred/jobcontrol/Job;)Ljava/lang/String;
> at
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:261)
> at
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:180)
> at org.apache.pig.PigServer.launchPlan(PigServer.java:1270)
> at
> org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1255)
> at org.apache.pig.PigServer.storeEx(PigServer.java:952)
> at org.apache.pig.PigServer.store(PigServer.java:919)
> at org.apache.pig.PigServer.openIterator(PigServer.java:832)
> at
> org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:682)
> at
>
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303)
> at
>
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:189)
> at
>
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
> at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
That's because the default pig jar file are compiled against hadoop-1.0.x,
but you're running it against hadoop-2.0.x. You need to re-build the pig
jar file.
Please do the following: (I am assuming that you're the installed hadoop
and pig-withouthadoop.jar.)
1) cd <pig home directory> (assuming that you have source code as well as
binaries.)
2) mv pig-0.10.0-withouthadoop.jar pig-0.10.0-withouthadoop.jar.bk
3) ant clean jar-withouthadoop -Dhadoopversion=23
You will see a new pig-0.10.0-withouthadoop.jar is built in the current
directory. Now you will no longer the error.
Thanks,
Cheolsoo
On Thu, Oct 25, 2012 at 10:14 PM, Nishant Neeraj <
[EMAIL PROTECTED]> wrote:
> I am new to Hadoop and Pig.
> I am trying to get Pig 0.10.0 [1] and Hadoop 2.0.2 [2] working together. I
> get
>
> java.lang.NoSuchMethodError:
>
> org.apache.hadoop.mapred.jobcontrol.JobControl.addJob(Lorg/apache/hadoop/mapred/jobcontrol/Job;)Ljava/lang/String;
>
> error, see complete stack trace in footnote [3]. How can I fix this?
>
> Here is what I am doing:
>
> grunt> cd hdfs:///data/
> grunt> cat test.data
> 2012-10-26 10:38:15,110 [main] WARN
> org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> 0 Wed Oct 24 14:35:21 IST 2012 name
> 1 Wed Oct 24 14:35:26 IST 2012 name
> 2 Wed Oct 24 14:35:31 IST 2012 name
> 3 Wed Oct 24 14:35:36 IST 2012 name
> 4 Wed Oct 24 14:35:41 IST 2012 name
> 5 Wed Oct 24 14:35:46 IST 2012 name
> 6 Wed Oct 24 14:35:51 IST 2012 name
> 7 Wed Oct 24 14:35:56 IST 2012 name
> 8 Wed Oct 24 14:36:01 IST 2012 name
> 9 Wed Oct 24 14:36:06 IST 2012 name
>
> grunt> data = LOAD 'test.data' ;
> 2012-10-26 10:38:29,389 [main] WARN org.apache.hadoop.conf.Configuration -
> mapred.jobtracker.maxtasks.per.job is deprecated. Instead, use
> mapreduce.jobtracker.maxtasks.perjob
> [ -- snip --]
>
> grunt> name = FILTER data BY $0 MATCHES '.*name.*' ;
> grunt> DUMP name
> [ -- snip -- ]
> 2012-10-26 10:39:23,299 [main] ERROR org.apache.pig.tools.grunt.Grunt -
> ERROR 2998: Unhandled internal error.
>
> org.apache.hadoop.mapred.jobcontrol.JobControl.addJob(Lorg/apache/hadoop/mapred/jobcontrol/Job;)Ljava/lang/String;
>
>
> ----
> [1]
> Apache Pig version 0.10.0 (r1328203)
> compiled Apr 19 2012, 22:54:12
>
> [2]
> Hadoop 2.0.2-alpha
> Subversion
>
> https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2.0.2-alpha/hadoop-common-project/hadoop-common-r
> 1392682
> Compiled by hortonmu on Tue Oct 2 00:44:10 UTC 2012
> From source with checksum efbdb59af73bfc103f1945d65dbf3071
>
> [3]
> Pig Stack Trace
> ---------------
> ERROR 2998: Unhandled internal error.
>
> org.apache.hadoop.mapred.jobcontrol.JobControl.addJob(Lorg/apache/hadoop/mapred/jobcontrol/Job;)Ljava/lang/String;
>
> java.lang.NoSuchMethodError:
>
> org.apache.hadoop.mapred.jobcontrol.JobControl.addJob(Lorg/apache/hadoop/mapred/jobcontrol/Job;)Ljava/lang/String;
> at
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:261)
> at
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:180)
> at org.apache.pig.PigServer.launchPlan(PigServer.java:1270)
> at
> org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1255)
> at org.apache.pig.PigServer.storeEx(PigServer.java:952)
> at org.apache.pig.PigServer.store(PigServer.java:919)
> at org.apache.pig.PigServer.openIterator(PigServer.java:832)
> at
> org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:682)
> at
>
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303)
> at
>
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:189)
> at
>
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
> at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
+
Nishant Neeraj 2012-10-26, 08:03
