Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Pig 0.10.0 and Hadoop 2.0.2: String JobControl.addJob(Job)


Copy link to this message
-
Re: Pig 0.10.0 and Hadoop 2.0.2: String JobControl.addJob(Job)
Hi Nishant,

That's because the default pig jar file are compiled against hadoop-1.0.x,
but you're running it against hadoop-2.0.x. You need to re-build the pig
jar file.

Please do the following: (I am assuming that you're the installed hadoop
and pig-withouthadoop.jar.)

1) cd <pig home directory> (assuming that you have source code as well as
binaries.)
2) mv pig-0.10.0-withouthadoop.jar pig-0.10.0-withouthadoop.jar.bk
3) ant clean jar-withouthadoop -Dhadoopversion=23

You will see a new pig-0.10.0-withouthadoop.jar is built in the current
directory. Now you will no longer the error.

Thanks,
Cheolsoo

On Thu, Oct 25, 2012 at 10:14 PM, Nishant Neeraj <
[EMAIL PROTECTED]> wrote:

> I am new to Hadoop and Pig.
> I am trying to get Pig 0.10.0 [1] and Hadoop 2.0.2 [2] working together. I
> get
>
>     java.lang.NoSuchMethodError:
>
> org.apache.hadoop.mapred.jobcontrol.JobControl.addJob(Lorg/apache/hadoop/mapred/jobcontrol/Job;)Ljava/lang/String;
>
> error, see complete stack trace in footnote [3]. How can I fix this?
>
> Here is what I am doing:
>
> grunt> cd hdfs:///data/
> grunt> cat test.data
> 2012-10-26 10:38:15,110 [main] WARN
>  org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> 0 Wed Oct 24 14:35:21 IST 2012 name
> 1 Wed Oct 24 14:35:26 IST 2012 name
> 2 Wed Oct 24 14:35:31 IST 2012 name
> 3 Wed Oct 24 14:35:36 IST 2012 name
> 4 Wed Oct 24 14:35:41 IST 2012 name
> 5 Wed Oct 24 14:35:46 IST 2012 name
> 6 Wed Oct 24 14:35:51 IST 2012 name
> 7 Wed Oct 24 14:35:56 IST 2012 name
> 8 Wed Oct 24 14:36:01 IST 2012 name
> 9 Wed Oct 24 14:36:06 IST 2012 name
>
> grunt> data = LOAD 'test.data' ;
> 2012-10-26 10:38:29,389 [main] WARN  org.apache.hadoop.conf.Configuration -
> mapred.jobtracker.maxtasks.per.job is deprecated. Instead, use
> mapreduce.jobtracker.maxtasks.perjob
> [ -- snip --]
>
> grunt> name = FILTER data BY $0 MATCHES '.*name.*' ;
> grunt> DUMP name
> [ -- snip -- ]
> 2012-10-26 10:39:23,299 [main] ERROR org.apache.pig.tools.grunt.Grunt -
> ERROR 2998: Unhandled internal error.
>
> org.apache.hadoop.mapred.jobcontrol.JobControl.addJob(Lorg/apache/hadoop/mapred/jobcontrol/Job;)Ljava/lang/String;
>
>
> ----
> [1]
> Apache Pig version 0.10.0 (r1328203)
> compiled Apr 19 2012, 22:54:12
>
> [2]
> Hadoop 2.0.2-alpha
> Subversion
>
> https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2.0.2-alpha/hadoop-common-project/hadoop-common-r
> 1392682
> Compiled by hortonmu on Tue Oct  2 00:44:10 UTC 2012
> From source with checksum efbdb59af73bfc103f1945d65dbf3071
>
> [3]
> Pig Stack Trace
> ---------------
> ERROR 2998: Unhandled internal error.
>
> org.apache.hadoop.mapred.jobcontrol.JobControl.addJob(Lorg/apache/hadoop/mapred/jobcontrol/Job;)Ljava/lang/String;
>
> java.lang.NoSuchMethodError:
>
> org.apache.hadoop.mapred.jobcontrol.JobControl.addJob(Lorg/apache/hadoop/mapred/jobcontrol/Job;)Ljava/lang/String;
>     at
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:261)
>     at
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:180)
>     at org.apache.pig.PigServer.launchPlan(PigServer.java:1270)
>     at
> org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1255)
>     at org.apache.pig.PigServer.storeEx(PigServer.java:952)
>     at org.apache.pig.PigServer.store(PigServer.java:919)
>     at org.apache.pig.PigServer.openIterator(PigServer.java:832)
>     at
> org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:682)
>     at
>
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303)
>     at
>
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:189)
>     at
>
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
>     at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB