Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig, mail # user - Pigunit Failure 10.1 running locally


+
j.barrett Strausser 2013-04-10, 14:05
+
Prashant Kommireddi 2013-04-10, 15:37
Copy link to this message
-
Re: Pigunit Failure 10.1 running locally
j.barrett Strausser 2013-04-10, 15:46
Yeah, I think for some reason, the pig version I was using had a dependency
on hadoop-core that had not been baked into hadoop common, but I'm not
really sure as I was piecing together some backups to get back to working.

I just sorted out all the dependencies issues a moment ago.

Below is my build.sbt file. If you aren't using scala it might not make
sense, but its easy enough to read off. I can confirm that all the versions
listed there are compatible with pigunit built from the 0.10.1 release.

https://github.com/bearrito/scala-pig/commit/4d287a535ac9d395a72f2fc8f53d9b66737d010c
All this headache is what I get for not pushing my work upstream.

Thanks,

barrett
On Wed, Apr 10, 2013 at 11:37 AM, Prashant Kommireddi
<[EMAIL PROTECTED]>wrote:

> What Hadoop version are you trying to use? Looks like you have 2
> different versions (1.X and 2.X) listed as dependencies.
>
> Sent from my iPhone
>
> On Apr 10, 2013, at 7:06 AM, "j.barrett Strausser"
> <[EMAIL PROTECTED]> wrote:
>
> > Greetings all,
> >
> > I am trying to run Pigunit and receiving an error. I had this previously
> > working, but had to rebuild my local workstation and didn't have
> everything
> > I should have had checked in. This is too deep into pig/hadoop for me to
> > effectively debug.
> >
> > I'm using scala so my versions/dependencies look like :
> >
> > libraryDependencies += "org.apache.pig" % "pig" % "0.10.1"
> >
> > libraryDependencies += "org.apache.hadoop" % "hadoop-common" %
> "2.0.3-alpha"
> >
> > libraryDependencies += "org.apache.hadoop" % "hadoop-core" % "1.1.2"
> >
> > I built the pigunit.jar file by hand as instructed in the docs.
> >
> > Any thoughts?
> >
> > The error is :
> >
> > java.lang.VerifyError: (class:
> > org/apache/hadoop/mapred/JobTrackerInstrumentation, method: create
> > signature:
> >
> (Lorg/apache/hadoop/mapred/JobTracker;Lorg/apache/hadoop/mapred/JobConf;)Lorg/apache/hadoop/mapred/JobTrackerInstrumentation;)
> > Incompatible argument to function
> >
> >
> > The stack trace is :
> >
> > [info]   at
> > org.apache.hadoop.mapred.LocalJobRunner.<init>(LocalJobRunner.java:420)
> > [info]   at org.apache.hadoop.mapred.JobClient.init(JobClient.java:488)
> > [info]   at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:473)
> > [info]   at
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:156)
> > [info]   at org.apache.pig.PigServer.launchPlan(PigServer.java:1279)
> > [info]   at
> > org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1264)
> > [info]   at org.apache.pig.PigServer.storeEx(PigServer.java:961)
> > [info]   at org.apache.pig.PigServer.store(PigServer.java:928)
> > [info]   at org.apache.pig.PigServer.openIterator(PigServer.java:841)
> > [info]   at org.apache.pig.pigunit.PigTest.getAlias(PigTest.java:183)
> > [info]   ...
> >
> >
> >
> >
> >
> > --
> >
> >
> > https://github.com/bearrito
> > @barrettsmash
>

--
https://github.com/bearrito
@barrettsmash
+
MARCOS MEDRADO RUBINELLI 2013-04-10, 15:39
+
j.barrett Strausser 2013-04-10, 17:25