Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - NoClassDefFoundError: org/apache/hadoop/yarn/service/CompositeService


Copy link to this message
-
RE: NoClassDefFoundError: org/apache/hadoop/yarn/service/CompositeService
Devaraj k 2013-07-11, 01:28
Hi Libo,

MRAppMaster is not able to load the yarn related jar files.

     Is this the classpath using by MRAppMaster or any other process?

"/opt/hadoop/hadoop-2.0.0-cdh4.3.0/conf:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/*:/contrib/capacity-scheduler/*.jar:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce2/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce/*"

If this is not the MRAppMaster process classpath, can you try to get the MRAppMaster java process classpath.
Thanks
Devaraj k
From: Yu, Libo [mailto:[EMAIL PROTECTED]]
Sent: 11 July 2013 01:30
To: '[EMAIL PROTECTED]'
Subject: NoClassDefFoundError: org/apache/hadoop/yarn/service/CompositeService

Hi,

I tried to run the wordcount example with yarn. Here is the command line:
hadoop jar share/hadoop/mapreduce2/hadoop-mapreduce-examples-2.0.0-cdh4.3.0.jar wordcount /user/lyu/wordcount/input /user/lyu/wordcount/output

But I got this exception:

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/yar
n/service/CompositeService
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
        at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.yarn.service.CompositeService
        at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
        ... 12 more
Could not find the main class: org.apache.hadoop.mapreduce.v2.app.MRAppMaster.  Program will exit

Here is conent of the CLASSPATH used by java:

/opt/hadoop/hadoop-2.0.0-cdh4.3.0/conf:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/*:/contrib/capacity-scheduler/*.jar:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce2/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce/*

Here is the console output:

13/07/09 20:43:07 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
13/07/09 20:43:07 INFO service.AbstractService: Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited.
13/07/09 20:43:07 INFO service.AbstractService: Service:org.apache.hadoop.yarn.client.YarnClientImpl is started.
13/07/09 20:43:08 INFO input.FileInputFormat: Total input paths to process : 0
13/07/09 20:43:08 INFO mapreduce.JobSubmitter: number of splits:0
13/07/09 20:43:08 WARN conf.Configuration: mapred.jar is deprecated. Instead, use mapreduce.job.jar
13/07/09 20:43:08 WARN conf.Configuration: mapred.output.value.class is deprecated. Instead, use mapreduce.job.output.value.class
13/07/09 20:43:08 WARN conf.Configuration: mapreduce.combine.class is deprecated. Instead, use mapreduce.job.combine.class
13/07/09 20:43:08 WARN conf.Configuration: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
13/07/09 20:43:08 WARN conf.Configuration: mapred.job.name is deprecated. Instead, use mapreduce.job.name
13/07/09 20:43:08 WARN conf.Configuration: mapreduce.reduce.class is deprecated. Instead, use mapreduce.job.reduce.class
13/07/09 20:43:08 WARN conf.Configuration: mapred.input.dir is deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
13/07/09 20:43:08 WARN conf.Configuration: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
13/07/09 20:43:08 WARN conf.Configuration: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
13/07/09 20:43:08 WARN conf.Configuration: mapred.output.key.class is deprecated. Instead, use mapreduce.job.output.key.class
13/07/09 20:43:08 WARN conf.Configuration: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
13/07/09 20:43:09 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1373416798858_0001
13/07/09 20:43:09 INFO client.YarnClientImpl: Submitted application application_1373416798858_0001 to ResourceManager at host1.libo.com/192.168.10.1:8032
13/07/09 20:43:09 INFO mapreduce.Job: The url to track the job: http://host1.libo.com:8088/proxy/application_1373416798858_0001/
13/07/09 20:43:09 INFO mapreduce.Job: Running job: job_1373416798858_0001
13/07/09 20:43:12 INFO mapreduce.Job: Job job_1373416798858_0001 running in uber mode : false
13/07/09 20:43:12 INFO mapreduce.Job:  map 0% reduce 0%
13/07/09 20:43:12 INFO mapreduce.Job: Job job_1373416798858