Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Collision with Hadoop (1.0.4) libs?


Copy link to this message
-
Collision with Hadoop (1.0.4) libs?
Hi,

 

I created "fat jar" to run my M/R "driver" application, and this fat jar
contains beside other libs:

slf4j-api-1.7.5.jar

slf4j-simple-1.7.5.jar

. and to delegate all commons-logging calls to slf4j.

jcl-over-slf4j-1.7.5.jar  

 

Unfortunatelly, when I start my application using jar command:

"hadoop jar myfat.jar ."

I get following:

 

13/04/19 22:50:21 INFO property.AppProps: using app.id:
F4278122BFBA5B98991997F8B15E68F6

Exception in thread "main" java.lang.NoSuchMethodError:
org.slf4j.spi.LocationAwareLogger.log(Lorg/slf4j/Marker;Ljava/lang/String;IL
java/lang/String;[Ljava/lang/Object;Ljava/lang/Throwable;)V

            at
org.apache.commons.logging.impl.SLF4JLocationAwareLog.debug(SLF4JLocationAwa
reLog.java:133)

            at
org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:
139)

            at
org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformat
ion.java:205)

            at
org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupI
nformation.java:184)

            at
org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupI
nformation.java:236)

            at
org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInform
ation.java:466)

            at
org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInfo
rmation.java:452)

            at
org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:1494)

            at
org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1395)

            at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:254)

            at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:123)

            at cascading.tap.hadoop.Hfs.getDefaultFileSystem(Hfs.java:307)

 

It seems that my application uses my bundled "jcl-over-slf4j-1.7.5.jar" to
delegate calls to some older slf4j-api.jar (not my bundled 1.7.5 version of
SLF4J), and I guess it can only be "slf4j-api-1.4.3.jar" found under
<hadoop_home>/lib ? That means that this old hadoop's slf4j lib has
preedence over the one that came bundled within my fat jar?!

 

Any suggestion to resolve this?

 

Regards,

Vjeran