Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> NoClassDefFoundError: org/apache/hadoop/yarn/service/CompositeService


Copy link to this message
-
RE: NoClassDefFoundError: org/apache/hadoop/yarn/service/CompositeService
Hi Libo,

MRAppMaster is not able to load the yarn related jar files.

     Is this the classpath using by MRAppMaster or any other process?

"/opt/hadoop/hadoop-2.0.0-cdh4.3.0/conf:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/*:/contrib/capacity-scheduler/*.jar:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce2/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce/*"

If this is not the MRAppMaster process classpath, can you try to get the MRAppMaster java process classpath.
Thanks
Devaraj k
From: Yu, Libo [mailto:[EMAIL PROTECTED]]
Sent: 11 July 2013 01:30
To: '[EMAIL PROTECTED]'
Subject: NoClassDefFoundError: org/apache/hadoop/yarn/service/CompositeService

Hi,

I tried to run the wordcount example with yarn. Here is the command line:
hadoop jar share/hadoop/mapreduce2/hadoop-mapreduce-examples-2.0.0-cdh4.3.0.jar wordcount /user/lyu/wordcount/input /user/lyu/wordcount/output

But I got this exception:

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/yar
n/service/CompositeService
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
        at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.yarn.service.CompositeService
        at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
        ... 12 more
Could not find the main class: org.apache.hadoop.mapreduce.v2.app.MRAppMaster.  Program will exit

Here is conent of the CLASSPATH used by java:

/opt/hadoop/hadoop-2.0.0-cdh4.3.0/conf:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/*:/contrib/capacity-scheduler/*.jar:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce2/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce/*

Here is the console output:

13/07/09 20:43:07 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
13/07/09 20:43:07 INFO service.AbstractService: Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited.
13/07/09 20:43:07 INFO service.AbstractService: Service:org.apache.hadoop.yarn.client.YarnClientImpl is started.
13/07/09 20:43:08 INFO input.FileInputFormat: Total input paths to process : 0
13/07/09 20:43:08 INFO mapreduce.JobSubmitter: number of splits:0
13/07/09 20:43:08 WARN conf.Configuration: mapred.jar is deprecated. Instead, use mapreduce.job.jar
13/07/09 20:43:08 WARN conf.Configuration: mapred.output.value.class is deprecated. Instead, use mapreduce.job.output.value.class
13/07/09 20:43:08 WARN conf.Configuration: mapreduce.combine.class is deprecated. Instead, use mapreduce.job.combine.class
13/07/09 20:43:08 WARN conf.Configuration: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
13/07/09 20:43:08 WARN conf.Configuration: mapred.job.name is deprecated. Instead, use mapreduce.job.name
13/07/09 20:43:08 WARN conf.Configuration: mapreduce.reduce.class is deprecated. Instead, use mapreduce.job.reduce.class
13/07/09 20:43:08 WARN conf.Configuration: mapred.input.dir is deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
13/07/09 20:43:08 WARN conf.Configuration: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
13/07/09 20:43:08 WARN conf.Configuration: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
13/07/09 20:43:08 WARN conf.Configuration: mapred.output.key.class is deprecated. Instead, use mapreduce.job.output.key.class
13/07/09 20:43:08 WARN conf.Configuration: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
13/07/09 20:43:09 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1373416798858_0001
13/07/09 20:43:09 INFO client.YarnClientImpl: Submitted application application_1373416798858_0001 to ResourceManager at host1.libo.com/192.168.10.1:8032
13/07/09 20:43:09 INFO mapreduce.Job: The url to track the job: http://host1.libo.com:8088/proxy/application_1373416798858_0001/
13/07/09 20:43:09 INFO mapreduce.Job: Running job: job_1373416798858_0001
13/07/09 20:43:12 INFO mapreduce.Job: Job job_1373416798858_0001 running in uber mode : false
13/07/09 20:43:12 INFO mapreduce.Job:  map 0% reduce 0%
13/07/09 20:43:12 INFO mapreduce.Job: Job job_1373416798858
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB