Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # dev >> Not able to execute MRv2


Copy link to this message
-
Re: Not able to execute MRv2
Hi,

For some reason none of the .log files are getting generated, so I had to
put some println and figure out the problems. I just see the .out files for
the RM, NM and the HistoryServer.

1)  I get the below error while starting the ResourceManager. In the
hadoop-mapreduce-1.0-SNAPSHOT/lib folder, there were avro-1.3.2.jar and
avro-1.4.1.jar files. Removing the 1.3.2 version of avro fixed the problem.
When I run JPS, I see RM, NM and HistoryServer.

java.lang.NoSuchMethodError: org.apache.avro.ipc.Server.start()V
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService.start(ResourceTrackerService.java:128)
at
org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68)

log4j:WARN No appenders could be found for logger
(org.apache.hadoop.metrics2.lib.MutableMetricsFactory).
log4j:WARN Please initialize the log4j system properly.

2) To run the sample program, had to create symbolic links for some of the
jars in the
/home/praveensripati/Hadoop/trunk/hadoop-common/target/hadoop-common-0.23.0-SNAPSHOT-bin/share/hadoop/common

When I ran the "hadoop classpath" command, it had some of the jars in the
above folder.

What would be the reason for the log files not being generated? I would like
to see the .log files to see the flow of the MRv2.

Thanks,
Praveen

On Tue, Aug 9, 2011 at 8:57 AM, Praveen Sripati <[EMAIL PROTECTED]>wrote:

>
> Hi,
>
> Looks like the below instructions are a bit outdated. I got the mapreduce
> code from the MR-279 branch and the rest of the code from trunk. The
> hadoop-mapreduce-1.0-SNAPSHOT-all.tar.gz file got generated successfully.
>
>
> http://svn.apache.org/repos/asf/hadoop/common/branches/MR-279/mapreduce/INSTALL
>
> 1) Expanded the tar file to the $YARN_INSTALL directory, created symbolic
> links to the jars in the hadoop-common/lib folder and ran the
> "bin/yarn-daemon.sh start resourcemanager" command from the $YARN_INSTALL
> directory to get the below error
>
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/conf/Configuration
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.conf.Configuration
>     at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>     at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>     at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>     at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> Could not find the main class:
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.  Program will
> exit.
>
> Then copied the hadoop-common-0.23.0-SNAPSHOT.jar from the
> hadoop-common/target to the hadoop-common/lib folder. Then the
> yarn-praveensripati-resourcemanager-praveensripati.out had the following
> error after starting the ResourceManager.
>
> Exception in thread "main" java.lang.IllegalStateException: For this
> operation, current State must be STARTED instead of INITED
>         at
> org.apache.hadoop.yarn.service.AbstractService.ensureCurrentState(AbstractService.java:101)
>         at
> org.apache.hadoop.yarn.service.AbstractService.stop(AbstractService.java:69)
>         at
> org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher.stop(ApplicationMasterLauncher.java:90)
>         at
> org.apache.hadoop.yarn.service.CompositeService.stop(CompositeService.java:89)
>         at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.stop(ResourceManager.java:423)
>         at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:493)
>
> Started the NodeManager and the HistoryServer. Running the jps shows only
> the ResourceManager and the NodeManager and not the HistoryServer.
>
> 2) From the hadoop-common/target/hadoop-common-0.23.0-SNAPSHOT-bin/bin
> folder I get the following error running the sample program
>
> ./hadoop jar
> $HADOOP_MAPRED_HOME/build/hadoop-mapred-examples-0.23.0-SNAPSHOT.jar