Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - Running Distributed shell in hadoop0.23


Copy link to this message
-
Re: Running Distributed shell in hadoop0.23
Hitesh Shah 2011-12-15, 05:09
The yarn jars are likely missing from the classpath. Could you try creating the symlinks as per step 11 from
http://svn.apache.org/viewvc/hadoop/common/trunk/hadoop-mapreduce-project/INSTALL?revision=1166955 ?

-- Hitesh

On Dec 14, 2011, at 8:35 PM, raghavendhra rahul wrote:

> I get the following erroer by the given command to run distributed shell
> hadoop1@master:~/hadoop/bin$ ./hadoop jar ../modules/hadoop-yarn-applications-distributedshell-0.23.0.jar org.apache.hadoop.yarn.applications.distributedshell.Client --jar ../modules/hadoop-yarn-applications-distributedshell-0.23.0.jar --shell_command ls --num_containers 5 --debug
> 2011-12-15 10:04:41,605 FATAL distributedshell.Client (Client.java:main(190)) - Error running CLient
> java.lang.NoClassDefFoundError: org/apache/hadoop/yarn/ipc/YarnRPC
>     at org.apache.hadoop.yarn.applications.distributedshell.Client.<init>(Client.java:206)
>     at org.apache.hadoop.yarn.applications.distributedshell.Client.main(Client.java:182)
>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>     at java.lang.reflect.Method.invoke(Method.java:616)
>     at org.apache.hadoop.util.RunJar.main(RunJar.java:189)
> Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.yarn.ipc.YarnRPC
>     at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
>     at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
>     at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
>     ... 7 more
>
>
> On Thu, Dec 15, 2011 at 12:09 AM, Hitesh Shah <[EMAIL PROTECTED]> wrote:
> Assuming you have a non-secure cluster setup ( the code does not handle security properly yet ), the following command would run the ls command on 5 allocated containers.
>
> $HADOOP_COMMON_HOME/bin/hadoop jar < path to hadoop-yarn-applications-distributedshell-0.24.0-SNAPSHOT.jar> org.apache.hadoop.yarn.applications.distributedshell.Client --jar < path to hadoop-yarn-applications-distributedshell-0.24.0-SNAPSHOT.jar> --shell_command ls --num_containers 5 --debug
>
> What the above does is upload the jar that contains the AppMaster class to hdfs, submits a new application request to launch the distributed shell app master on a container which then in turn runs the shell command on the no. of containers specified.
>
> -- Hitesh
>
> On Dec 14, 2011, at 1:06 AM, sri ram wrote:
>
> > Hi,
> >      Can anyone give the procedure about how to run Distibuted shell example in hadoop yarn.So that i try to understand how applicatin master really works.
>
>