Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # dev >> Not able to execute MRv2


Copy link to this message
-
Not able to execute MRv2
Hi,

Looks like the below instructions are a bit outdated. I got the mapreduce
code from the MR-279 branch and the rest of the code from trunk. The
hadoop-mapreduce-1.0-SNAPSHOT-all.tar.gz file got generated successfully.

http://svn.apache.org/repos/asf/hadoop/common/branches/MR-279/mapreduce/INSTALL

1) Expanded the tar file to the $YARN_INSTALL directory, created symbolic
links to the jars in the hadoop-common/lib folder and ran the
"bin/yarn-daemon.sh start resourcemanager" command from the $YARN_INSTALL
directory to get the below error

Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/hadoop/conf/Configuration
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.conf.Configuration
    at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
Could not find the main class:
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.  Program will
exit.

Then copied the hadoop-common-0.23.0-SNAPSHOT.jar from the
hadoop-common/target to the hadoop-common/lib folder. Then the
yarn-praveensripati-resourcemanager-praveensripati.out had the following
error after starting the ResourceManager.

Exception in thread "main" java.lang.IllegalStateException: For this
operation, current State must be STARTED instead of INITED
        at
org.apache.hadoop.yarn.service.AbstractService.ensureCurrentState(AbstractService.java:101)
        at
org.apache.hadoop.yarn.service.AbstractService.stop(AbstractService.java:69)
        at
org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher.stop(ApplicationMasterLauncher.java:90)
        at
org.apache.hadoop.yarn.service.CompositeService.stop(CompositeService.java:89)
        at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.stop(ResourceManager.java:423)
        at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:493)

Started the NodeManager and the HistoryServer. Running the jps shows only
the ResourceManager and the NodeManager and not the HistoryServer.

2) From the hadoop-common/target/hadoop-common-0.23.0-SNAPSHOT-bin/bin
folder I get the following error running the sample program

./hadoop jar
$HADOOP_MAPRED_HOME/build/hadoop-mapred-examples-0.23.0-SNAPSHOT.jar
randomwriter -Dmapreduce.job.user.name=$USER
-Dmapreduce.randomwriter.bytespermap=10000 -Ddfs.blocksize=536870912
-Ddfs.block.size=536870912 -libjars
$YARN_INSTALL/modules/hadoop-mapreduce-client-jobclient-1.0-SNAPSHOT.jar
output

java.lang.NoClassDefFoundError:
org/apache/hadoop/mapreduce/lib/partition/InputSampler$Sampler
    at java.lang.Class.getDeclaredMethods0(Native Method)
    at java.lang.Class.privateGetDeclaredMethods(Class.java:2427)
    at java.lang.Class.getMethod0(Class.java:2670)
    at java.lang.Class.getMethod(Class.java:1603)
    at
org.apache.hadoop.util.ProgramDriver$ProgramDescription.<init>(ProgramDriver.java:60)

The hadoop-mapreduce-client-core-1.0-SNAPSHOT.jar which has the
InputSampler.java class is there in the hadoop-common/lib folder, but still
getting the above error.

None of the .log files are getting generated. So, it's getting difficult yo
debug. Appreciate help in resolving this.

Thanks,
Praveen
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB