Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # dev >> Not able to execute MRv2


Copy link to this message
-
Re: Not able to execute MRv2
Hi,

For some reason none of the .log files are getting generated, so I had to
put some println and figure out the problems. I just see the .out files for
the RM, NM and the HistoryServer.

1)  I get the below error while starting the ResourceManager. In the
hadoop-mapreduce-1.0-SNAPSHOT/lib folder, there were avro-1.3.2.jar and
avro-1.4.1.jar files. Removing the 1.3.2 version of avro fixed the problem.
When I run JPS, I see RM, NM and HistoryServer.

java.lang.NoSuchMethodError: org.apache.avro.ipc.Server.start()V
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService.start(ResourceTrackerService.java:128)
at
org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68)

log4j:WARN No appenders could be found for logger
(org.apache.hadoop.metrics2.lib.MutableMetricsFactory).
log4j:WARN Please initialize the log4j system properly.

2) To run the sample program, had to create symbolic links for some of the
jars in the
/home/praveensripati/Hadoop/trunk/hadoop-common/target/hadoop-common-0.23.0-SNAPSHOT-bin/share/hadoop/common

When I ran the "hadoop classpath" command, it had some of the jars in the
above folder.

What would be the reason for the log files not being generated? I would like
to see the .log files to see the flow of the MRv2.

Thanks,
Praveen

On Tue, Aug 9, 2011 at 8:57 AM, Praveen Sripati <[EMAIL PROTECTED]>wrote:

>
> Hi,
>
> Looks like the below instructions are a bit outdated. I got the mapreduce
> code from the MR-279 branch and the rest of the code from trunk. The
> hadoop-mapreduce-1.0-SNAPSHOT-all.tar.gz file got generated successfully.
>
>
> http://svn.apache.org/repos/asf/hadoop/common/branches/MR-279/mapreduce/INSTALL
>
> 1) Expanded the tar file to the $YARN_INSTALL directory, created symbolic
> links to the jars in the hadoop-common/lib folder and ran the
> "bin/yarn-daemon.sh start resourcemanager" command from the $YARN_INSTALL
> directory to get the below error
>
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/conf/Configuration
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.conf.Configuration
>     at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>     at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>     at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>     at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> Could not find the main class:
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.  Program will
> exit.
>
> Then copied the hadoop-common-0.23.0-SNAPSHOT.jar from the
> hadoop-common/target to the hadoop-common/lib folder. Then the
> yarn-praveensripati-resourcemanager-praveensripati.out had the following
> error after starting the ResourceManager.
>
> Exception in thread "main" java.lang.IllegalStateException: For this
> operation, current State must be STARTED instead of INITED
>         at
> org.apache.hadoop.yarn.service.AbstractService.ensureCurrentState(AbstractService.java:101)
>         at
> org.apache.hadoop.yarn.service.AbstractService.stop(AbstractService.java:69)
>         at
> org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher.stop(ApplicationMasterLauncher.java:90)
>         at
> org.apache.hadoop.yarn.service.CompositeService.stop(CompositeService.java:89)
>         at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.stop(ResourceManager.java:423)
>         at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:493)
>
> Started the NodeManager and the HistoryServer. Running the jps shows only
> the ResourceManager and the NodeManager and not the HistoryServer.
>
> 2) From the hadoop-common/target/hadoop-common-0.23.0-SNAPSHOT-bin/bin
> folder I get the following error running the sample program
>
> ./hadoop jar
> $HADOOP_MAPRED_HOME/build/hadoop-mapred-examples-0.23.0-SNAPSHOT.jar
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB