Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> Collision with Hadoop (1.0.4) libs?


Copy link to this message
-
Collision with Hadoop (1.0.4) libs?
Hi,

 

I created "fat jar" to run my M/R "driver" application, and this fat jar
contains beside other libs:

slf4j-api-1.7.5.jar

slf4j-simple-1.7.5.jar

. and to delegate all commons-logging calls to slf4j.

jcl-over-slf4j-1.7.5.jar  

 

Unfortunatelly, when I start my application using jar command:

"hadoop jar myfat.jar ."

I get following:

 

13/04/19 22:50:21 INFO property.AppProps: using app.id:
F4278122BFBA5B98991997F8B15E68F6

Exception in thread "main" java.lang.NoSuchMethodError:
org.slf4j.spi.LocationAwareLogger.log(Lorg/slf4j/Marker;Ljava/lang/String;IL
java/lang/String;[Ljava/lang/Object;Ljava/lang/Throwable;)V

            at
org.apache.commons.logging.impl.SLF4JLocationAwareLog.debug(SLF4JLocationAwa
reLog.java:133)

            at
org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:
139)

            at
org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformat
ion.java:205)

            at
org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupI
nformation.java:184)

            at
org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupI
nformation.java:236)

            at
org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInform
ation.java:466)

            at
org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInfo
rmation.java:452)

            at
org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:1494)

            at
org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1395)

            at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:254)

            at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:123)

            at cascading.tap.hadoop.Hfs.getDefaultFileSystem(Hfs.java:307)

 

It seems that my application uses my bundled "jcl-over-slf4j-1.7.5.jar" to
delegate calls to some older slf4j-api.jar (not my bundled 1.7.5 version of
SLF4J), and I guess it can only be "slf4j-api-1.4.3.jar" found under
<hadoop_home>/lib ? That means that this old hadoop's slf4j lib has
preedence over the one that came bundled within my fat jar?!

 

Any suggestion to resolve this?

 

Regards,

Vjeran

 

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB