Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Running Distributed shell in hadoop0.23


Copy link to this message
-
Re: Running Distributed shell in hadoop0.23
The yarn jars are likely missing from the classpath. Could you try creating the symlinks as per step 11 from
http://svn.apache.org/viewvc/hadoop/common/trunk/hadoop-mapreduce-project/INSTALL?revision=1166955 ?

-- Hitesh

On Dec 14, 2011, at 8:35 PM, raghavendhra rahul wrote:

> I get the following erroer by the given command to run distributed shell
> hadoop1@master:~/hadoop/bin$ ./hadoop jar ../modules/hadoop-yarn-applications-distributedshell-0.23.0.jar org.apache.hadoop.yarn.applications.distributedshell.Client --jar ../modules/hadoop-yarn-applications-distributedshell-0.23.0.jar --shell_command ls --num_containers 5 --debug
> 2011-12-15 10:04:41,605 FATAL distributedshell.Client (Client.java:main(190)) - Error running CLient
> java.lang.NoClassDefFoundError: org/apache/hadoop/yarn/ipc/YarnRPC
>     at org.apache.hadoop.yarn.applications.distributedshell.Client.<init>(Client.java:206)
>     at org.apache.hadoop.yarn.applications.distributedshell.Client.main(Client.java:182)
>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>     at java.lang.reflect.Method.invoke(Method.java:616)
>     at org.apache.hadoop.util.RunJar.main(RunJar.java:189)
> Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.yarn.ipc.YarnRPC
>     at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
>     at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
>     at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
>     ... 7 more
>
>
> On Thu, Dec 15, 2011 at 12:09 AM, Hitesh Shah <[EMAIL PROTECTED]> wrote:
> Assuming you have a non-secure cluster setup ( the code does not handle security properly yet ), the following command would run the ls command on 5 allocated containers.
>
> $HADOOP_COMMON_HOME/bin/hadoop jar < path to hadoop-yarn-applications-distributedshell-0.24.0-SNAPSHOT.jar> org.apache.hadoop.yarn.applications.distributedshell.Client --jar < path to hadoop-yarn-applications-distributedshell-0.24.0-SNAPSHOT.jar> --shell_command ls --num_containers 5 --debug
>
> What the above does is upload the jar that contains the AppMaster class to hdfs, submits a new application request to launch the distributed shell app master on a container which then in turn runs the shell command on the no. of containers specified.
>
> -- Hitesh
>
> On Dec 14, 2011, at 1:06 AM, sri ram wrote:
>
> > Hi,
> >      Can anyone give the procedure about how to run Distibuted shell example in hadoop yarn.So that i try to understand how applicatin master really works.
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB