Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> NoClassDefFoundError after upgrading to pig 0.10.0 from 0.9.0


+
Matthew Hayes 2012-06-08, 19:07
+
Dmitriy Ryaboy 2012-06-11, 00:05
+
Matthew Hayes 2012-06-11, 17:15
+
Matthew Hayes 2012-06-29, 17:36
Copy link to this message
-
Re: NoClassDefFoundError after upgrading to pig 0.10.0 from 0.9.0
Yep, you are right, the pom is not generated, but checked in
statically; looks like it's out of date. One more reason to mavenize
Pig.. can you open a Jira for this? We'll fix.

D

On Mon, Jun 11, 2012 at 10:15 AM, Matthew Hayes <[EMAIL PROTECTED]> wrote:
> What I mean is: are the versions in the pom for pig 0.10.0 inconsistent with the versions specified in the ivy file used to build pig?  To be clear, I am building a separate project, and I am getting pig and its dependencies using ivy.
>
> Looking in ivy.xml in the pig 0.10.0 release:
>
>     <dependency org="org.apache.avro" name="avro" rev="${avro.version}"
>       conf="compile->default;checkstyle->master"/>
>
> ...
>
>     <dependency org="org.codehaus.jackson" name="jackson-mapper-asl" rev="${jackson.version}"
>       conf="compile->master"/>
>     <dependency org="org.codehaus.jackson" name="jackson-core-asl" rev="${jackson.version}"
>       conf="compile->master"/>
>
> Where avro.version is avro.version=1.5.3 and jackson.version=1.7.3.
>
> However, in the pom.xml for pig 0.10.0:
>
>       <groupId>org.apache.hadoop</groupId>
>       <artifactId>avro</artifactId>
>       <version>1.3.2</version>
>
> And when I look up the pom for org.apache.hadoop's avro 1.3.2 in the central repository, I see a version of jackson inconsistent with what pig was compiled with:
>
>     <dependency>
>       <groupId>org.codehaus.jackson</groupId>
>       <artifactId>jackson-mapper-asl</artifactId>
>       <version>1.4.2</version>
>       <scope>compile</scope>
>     </dependency>
>
> It's 1.4.2, not 1.7.3.
>
> Am I doing something wrong here?  Below is my ivy.xml.  It's the same as what I used for 0.9.0 but I changed the pig version to 0.10.0.
>
> <ivy-module version="2.0">
>     <info organisation="datafu" module="datafu"/>
>     <dependencies>
>         <dependency org="org.apache.pig" name="pig" rev="0.10.0"/>
>         <dependency org="it.unimi.dsi" name="fastutil" rev="6.3"/>
>         <dependency org="joda-time" name="joda-time" rev="1.6"/>
>         <dependency org="org.apache.commons" name="commons-math" rev="2.1"/>
>         <dependency org="commons-io" name="commons-io" rev="1.4"/>
>         <dependency org="org.apache.hadoop" name="hadoop-core" rev="0.20.2"/>
>         <dependency org="org.testng" name="testng" rev="6.2"/>
>         <dependency org="com.google.guava" name="guava" rev="r06" />
>
>         <!-- needed for pigunit to work -->
>         <dependency org="log4j" name="log4j" rev="1.2.14" />
>         <dependency org="jline" name="jline" rev="0.9.94" />
>         <dependency org="org.antlr" name="antlr" rev="3.2" />
>     </dependencies>
> </ivy-module>
>
> Thanks,
> Matt
> ________________________________________
> From: Dmitriy Ryaboy [[EMAIL PROTECTED]]
> Sent: Sunday, June 10, 2012 5:05 PM
> To: [EMAIL PROTECTED]
> Subject: Re: NoClassDefFoundError after upgrading to pig 0.10.0 from 0.9.0
>
> I am not sure what you mean -- ivy pulls down 1.4.2 but pig builds from 1.7.3? Pig gets its jars from ivy, so it must be building from whatever ivy pulls down. Sounds like there is some packaging problem, but I am not sure how to interpret what you said.
>
> On Jun 8, 2012, at 12:07 PM, Matthew Hayes <[EMAIL PROTECTED]> wrote:
>
>> I'm using ivy to download dependencies for pig, but after updating the version to 0.10.0 I am getting errors in my unit tests:
>>
>>   [testng] java.lang.NoClassDefFoundError: org/codehaus/jackson/map/util/LRUMap
>>   [testng] at org.apache.pig.builtin.JsonMetadata.<init>(JsonMetadata.java:75)
>>   [testng] at org.apache.pig.builtin.PigStorage.getSchema(PigStorage.java:466)
>>   [testng] at org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:151)
>>   [testng] at org.apache.pig.newplan.logical.relational.LOLoad.getSchema(LOLoad.java:110)
>>   ...etc.
>>
>> It looks like the dependency on jackson-mapper-asl is coming from avro.  Looking in jackson-mapper-asl-1.7.3.jar I see the LRUMap class.  However using ivy to pull down dependencies I get jackson-mapper-asl-1.4.2.jar, not jackson-mapper-asl-1.7.3.jar.  When I build pig 0.10.0 from source it uses 1.7.3.
+
Gianmarco De Francisci Mo... 2012-07-02, 13:36
+
Matthew Hayes 2012-07-03, 06:22
+
Matthew Hayes 2012-07-05, 22:06