|
|
Cyril Bogus 2013-03-08, 18:46
I am trying to have a two node cluster. Hadoop 1.0.4
the master is under the account A the slave is under the account B
I run into an exception when I try to start the cluster. Here is my output
A@owner-5:~$ /home/hadoop/bin/start-all.sh Warning: $HADOOP_HOME is deprecated.
namenode running as process 3020. Stop it first. A@master: starting datanode, logging to /home/hadoop/libexec/../logs/hadoop-A-datanode-owner-5.out B@slave: Warning: $HADOOP_HOME is deprecated. B@slave: B@slave: starting datanode, logging to /home/owner/hadoop//logs/hadoop-owner-datanode-owner-7.out B@slave: Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/util/PlatformName B@slave: Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.util.PlatformName B@slave: at java.net.URLClassLoader$1.run(URLClassLoader.java:202) B@slave: at java.security.AccessController.doPrivileged(Native Method) B@slave: at java.net.URLClassLoader.findClass(URLClassLoader.java:190) B@slave: at java.lang.ClassLoader.loadClass(ClassLoader.java:306) B@slave: at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) B@slave: at java.lang.ClassLoader.loadClass(ClassLoader.java:247) B@slave: Could not find the main class: org.apache.hadoop.util.PlatformName. Program will exit. B@slave: Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/hdfs/server/datanode/DataNode A@master: secondarynamenode running as process 3352. Stop it first. starting jobtracker, logging to /home/hadoop/libexec/../logs/hadoop-A-jobtracker-owner-5.out B@slave: Warning: $HADOOP_HOME is deprecated. B@slave: B@slave: starting tasktracker, logging to /home/owner/hadoop//logs/hadoop-owner-tasktracker-owner-7.out A@master: starting tasktracker, logging to /home/hadoop/libexec/../logs/hadoop-A-tasktracker-owner-5.out B@slave: Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/util/PlatformName B@slave: Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.util.PlatformName B@slave: at java.net.URLClassLoader$1.run(URLClassLoader.java:202) B@slave: at java.security.AccessController.doPrivileged(Native Method) B@slave: at java.net.URLClassLoader.findClass(URLClassLoader.java:190) B@slave: at java.lang.ClassLoader.loadClass(ClassLoader.java:306) B@slave: at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) B@slave: at java.lang.ClassLoader.loadClass(ClassLoader.java:247) B@slave: Could not find the main class: org.apache.hadoop.util.PlatformName. Program will exit. B@slave: Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/mapred/TaskTracker A@owner-5:~$
-
Re: Small cluster setup.
Cyril Bogus 2013-03-13, 13:59
Thanks for the reply.
I am running on Linux. The problem IS the config file but since I couldn't figure out where, I made two copies of hadoop. One where it is looking for it, and where other nodes will be looking for it.
In my config file, everything is I have set hadoop in /home/hadoop but it is looking for it in /home/owner/hadoop (for some processes at least) and that was the problem. On Tue, Mar 12, 2013 at 11:06 AM, Jean-Marc Spaggiari < [EMAIL PROTECTED]> wrote:
> Hi Cyril, > > Are you running in Cygwin? Or Linux? Also, you might want to share > your configuration files if you want someone to take a look. > > JM > > 2013/3/8 Cyril Bogus <[EMAIL PROTECTED]>: > > I am trying to have a two node cluster. > > Hadoop 1.0.4 > > > > the master is under the account A > > the slave is under the account B > > > > I run into an exception when I try to start the cluster. > > Here is my output > > > > A@owner-5:~$ /home/hadoop/bin/start-all.sh > > Warning: $HADOOP_HOME is deprecated. > > > > namenode running as process 3020. Stop it first. > > A@master: starting datanode, logging to > > /home/hadoop/libexec/../logs/hadoop-A-datanode-owner-5.out > > B@slave: Warning: $HADOOP_HOME is deprecated. > > B@slave: > > B@slave: starting datanode, logging to > > /home/owner/hadoop//logs/hadoop-owner-datanode-owner-7.out > > B@slave: Exception in thread "main" java.lang.NoClassDefFoundError: > > org/apache/hadoop/util/PlatformName > > B@slave: Caused by: java.lang.ClassNotFoundException: > > org.apache.hadoop.util.PlatformName > > B@slave: at java.net.URLClassLoader$1.run(URLClassLoader.java:202) > > B@slave: at java.security.AccessController.doPrivileged(Native > Method) > > B@slave: at > java.net.URLClassLoader.findClass(URLClassLoader.java:190) > > B@slave: at java.lang.ClassLoader.loadClass(ClassLoader.java:306) > > B@slave: at > > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) > > B@slave: at java.lang.ClassLoader.loadClass(ClassLoader.java:247) > > B@slave: Could not find the main class: > org.apache.hadoop.util.PlatformName. > > Program will exit. > > B@slave: Exception in thread "main" java.lang.NoClassDefFoundError: > > org/apache/hadoop/hdfs/server/datanode/DataNode > > A@master: secondarynamenode running as process 3352. Stop it first. > > starting jobtracker, logging to > > /home/hadoop/libexec/../logs/hadoop-A-jobtracker-owner-5.out > > B@slave: Warning: $HADOOP_HOME is deprecated. > > B@slave: > > B@slave: starting tasktracker, logging to > > /home/owner/hadoop//logs/hadoop-owner-tasktracker-owner-7.out > > A@master: starting tasktracker, logging to > > /home/hadoop/libexec/../logs/hadoop-A-tasktracker-owner-5.out > > B@slave: Exception in thread "main" java.lang.NoClassDefFoundError: > > org/apache/hadoop/util/PlatformName > > B@slave: Caused by: java.lang.ClassNotFoundException: > > org.apache.hadoop.util.PlatformName > > B@slave: at java.net.URLClassLoader$1.run(URLClassLoader.java:202) > > B@slave: at java.security.AccessController.doPrivileged(Native > Method) > > B@slave: at > java.net.URLClassLoader.findClass(URLClassLoader.java:190) > > B@slave: at java.lang.ClassLoader.loadClass(ClassLoader.java:306) > > B@slave: at > > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) > > B@slave: at java.lang.ClassLoader.loadClass(ClassLoader.java:247) > > B@slave: Could not find the main class: > org.apache.hadoop.util.PlatformName. > > Program will exit. > > B@slave: Exception in thread "main" java.lang.NoClassDefFoundError: > > org/apache/hadoop/mapred/TaskTracker > > A@owner-5:~$ > > >
-
Re: Small cluster setup.
Nitin Pawar 2013-03-13, 14:02
Cyril,
how did you install hadoop? when you start hadoop ... do you start it from the location where it is installed or from users home directory?
try setting HADOOP_HOME (its deprecated but it helps to resolve issues like where the config files are located etc) On Wed, Mar 13, 2013 at 7:29 PM, Cyril Bogus <[EMAIL PROTECTED]> wrote:
> Thanks for the reply. > > I am running on Linux. The problem IS the config file but since I couldn't > figure out where, I made two copies of hadoop. One where it is looking for > it, and where other nodes will be looking for it. > > In my config file, everything is I have set hadoop in /home/hadoop but it > is looking for it in /home/owner/hadoop (for some processes at least) and > that was the problem. > > > On Tue, Mar 12, 2013 at 11:06 AM, Jean-Marc Spaggiari < > [EMAIL PROTECTED]> wrote: > >> Hi Cyril, >> >> Are you running in Cygwin? Or Linux? Also, you might want to share >> your configuration files if you want someone to take a look. >> >> JM >> >> 2013/3/8 Cyril Bogus <[EMAIL PROTECTED]>: >> > I am trying to have a two node cluster. >> > Hadoop 1.0.4 >> > >> > the master is under the account A >> > the slave is under the account B >> > >> > I run into an exception when I try to start the cluster. >> > Here is my output >> > >> > A@owner-5:~$ /home/hadoop/bin/start-all.sh >> > Warning: $HADOOP_HOME is deprecated. >> > >> > namenode running as process 3020. Stop it first. >> > A@master: starting datanode, logging to >> > /home/hadoop/libexec/../logs/hadoop-A-datanode-owner-5.out >> > B@slave: Warning: $HADOOP_HOME is deprecated. >> > B@slave: >> > B@slave: starting datanode, logging to >> > /home/owner/hadoop//logs/hadoop-owner-datanode-owner-7.out >> > B@slave: Exception in thread "main" java.lang.NoClassDefFoundError: >> > org/apache/hadoop/util/PlatformName >> > B@slave: Caused by: java.lang.ClassNotFoundException: >> > org.apache.hadoop.util.PlatformName >> > B@slave: at java.net.URLClassLoader$1.run(URLClassLoader.java:202) >> > B@slave: at java.security.AccessController.doPrivileged(Native >> Method) >> > B@slave: at >> java.net.URLClassLoader.findClass(URLClassLoader.java:190) >> > B@slave: at java.lang.ClassLoader.loadClass(ClassLoader.java:306) >> > B@slave: at >> > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) >> > B@slave: at java.lang.ClassLoader.loadClass(ClassLoader.java:247) >> > B@slave: Could not find the main class: >> org.apache.hadoop.util.PlatformName. >> > Program will exit. >> > B@slave: Exception in thread "main" java.lang.NoClassDefFoundError: >> > org/apache/hadoop/hdfs/server/datanode/DataNode >> > A@master: secondarynamenode running as process 3352. Stop it first. >> > starting jobtracker, logging to >> > /home/hadoop/libexec/../logs/hadoop-A-jobtracker-owner-5.out >> > B@slave: Warning: $HADOOP_HOME is deprecated. >> > B@slave: >> > B@slave: starting tasktracker, logging to >> > /home/owner/hadoop//logs/hadoop-owner-tasktracker-owner-7.out >> > A@master: starting tasktracker, logging to >> > /home/hadoop/libexec/../logs/hadoop-A-tasktracker-owner-5.out >> > B@slave: Exception in thread "main" java.lang.NoClassDefFoundError: >> > org/apache/hadoop/util/PlatformName >> > B@slave: Caused by: java.lang.ClassNotFoundException: >> > org.apache.hadoop.util.PlatformName >> > B@slave: at java.net.URLClassLoader$1.run(URLClassLoader.java:202) >> > B@slave: at java.security.AccessController.doPrivileged(Native >> Method) >> > B@slave: at >> java.net.URLClassLoader.findClass(URLClassLoader.java:190) >> > B@slave: at java.lang.ClassLoader.loadClass(ClassLoader.java:306) >> > B@slave: at >> > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) >> > B@slave: at java.lang.ClassLoader.loadClass(ClassLoader.java:247) >> > B@slave: Could not find the main class: >> org.apache.hadoop.util.PlatformName. >> > Program will exit. >> > B@slave: Exception in thread "main" java.lang.NoClassDefFoundError: Nitin Pawar
-
Re: Small cluster setup.
Cyril Bogus 2013-03-13, 14:21
Hi Nitin,
As part of my configuration I have set all the environment variables AND added HADOOP_PREFIX. But the problem still persist so I will just keep the extra copy in order to move forward.
And I start it from /home/hadoop. I have no idea why it is looking for /home/agnik/hadoop On Wed, Mar 13, 2013 at 10:03 AM, Nitin Pawar <[EMAIL PROTECTED]>wrote:
> or set this variable HADOOP_PREFIX to the directory where hadoop is > installed. > > > On Wed, Mar 13, 2013 at 7:32 PM, Nitin Pawar <[EMAIL PROTECTED]>wrote: > >> Cyril, >> >> how did you install hadoop? >> when you start hadoop ... do you start it from the location where it is >> installed or from users home directory? >> >> try setting HADOOP_HOME (its deprecated but it helps to resolve issues >> like where the config files are located etc) >> >> >> On Wed, Mar 13, 2013 at 7:29 PM, Cyril Bogus <[EMAIL PROTECTED]>wrote: >> >>> Thanks for the reply. >>> >>> I am running on Linux. The problem IS the config file but since I >>> couldn't figure out where, I made two copies of hadoop. One where it is >>> looking for it, and where other nodes will be looking for it. >>> >>> In my config file, everything is I have set hadoop in /home/hadoop but >>> it is looking for it in /home/owner/hadoop (for some processes at least) >>> and that was the problem. >>> >>> >>> On Tue, Mar 12, 2013 at 11:06 AM, Jean-Marc Spaggiari < >>> [EMAIL PROTECTED]> wrote: >>> >>>> Hi Cyril, >>>> >>>> Are you running in Cygwin? Or Linux? Also, you might want to share >>>> your configuration files if you want someone to take a look. >>>> >>>> JM >>>> >>>> 2013/3/8 Cyril Bogus <[EMAIL PROTECTED]>: >>>> > I am trying to have a two node cluster. >>>> > Hadoop 1.0.4 >>>> > >>>> > the master is under the account A >>>> > the slave is under the account B >>>> > >>>> > I run into an exception when I try to start the cluster. >>>> > Here is my output >>>> > >>>> > A@owner-5:~$ /home/hadoop/bin/start-all.sh >>>> > Warning: $HADOOP_HOME is deprecated. >>>> > >>>> > namenode running as process 3020. Stop it first. >>>> > A@master: starting datanode, logging to >>>> > /home/hadoop/libexec/../logs/hadoop-A-datanode-owner-5.out >>>> > B@slave: Warning: $HADOOP_HOME is deprecated. >>>> > B@slave: >>>> > B@slave: starting datanode, logging to >>>> > /home/owner/hadoop//logs/hadoop-owner-datanode-owner-7.out >>>> > B@slave: Exception in thread "main" java.lang.NoClassDefFoundError: >>>> > org/apache/hadoop/util/PlatformName >>>> > B@slave: Caused by: java.lang.ClassNotFoundException: >>>> > org.apache.hadoop.util.PlatformName >>>> > B@slave: at >>>> java.net.URLClassLoader$1.run(URLClassLoader.java:202) >>>> > B@slave: at java.security.AccessController.doPrivileged(Native >>>> Method) >>>> > B@slave: at >>>> java.net.URLClassLoader.findClass(URLClassLoader.java:190) >>>> > B@slave: at java.lang.ClassLoader.loadClass(ClassLoader.java:306) >>>> > B@slave: at >>>> > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) >>>> > B@slave: at java.lang.ClassLoader.loadClass(ClassLoader.java:247) >>>> > B@slave: Could not find the main class: >>>> org.apache.hadoop.util.PlatformName. >>>> > Program will exit. >>>> > B@slave: Exception in thread "main" java.lang.NoClassDefFoundError: >>>> > org/apache/hadoop/hdfs/server/datanode/DataNode >>>> > A@master: secondarynamenode running as process 3352. Stop it first. >>>> > starting jobtracker, logging to >>>> > /home/hadoop/libexec/../logs/hadoop-A-jobtracker-owner-5.out >>>> > B@slave: Warning: $HADOOP_HOME is deprecated. >>>> > B@slave: >>>> > B@slave: starting tasktracker, logging to >>>> > /home/owner/hadoop//logs/hadoop-owner-tasktracker-owner-7.out >>>> > A@master: starting tasktracker, logging to >>>> > /home/hadoop/libexec/../logs/hadoop-A-tasktracker-owner-5.out >>>> > B@slave: Exception in thread "main" java.lang.NoClassDefFoundError: >>>> > org/apache/hadoop/util/PlatformName >>>> > B@slave: Caused by: java.lang.ClassNotFoundException:
-
Re: Small cluster setup.
hadoop hive 2013-03-14, 13:53
check your hadoop-env.sh file, set hadoop_home path correctly.
On Wed, Mar 13, 2013 at 7:51 PM, Cyril Bogus <[EMAIL PROTECTED]> wrote:
> Hi Nitin, > > As part of my configuration I have set all the environment variables AND > added HADOOP_PREFIX. But the problem still persist so I will just keep the > extra copy in order to move forward. > > And I start it from /home/hadoop. I have no idea why it is looking for > /home/agnik/hadoop > > > On Wed, Mar 13, 2013 at 10:03 AM, Nitin Pawar <[EMAIL PROTECTED]>wrote: > >> or set this variabole HADOOP_PREFIX to the directory where hadoop is >> installed. >> >> >> On Wed, Mar 13, 2013 at 7:32 PM, Nitin Pawar <[EMAIL PROTECTED]>wrote: >> >>> Cyril, >>> >>> how did you install hadoop? >>> when you start hadoop ... do you start it from the location where it is >>> installed or from users home directory? >>> >>> try setting HADOOP_HOME (its deprecated but it helps to resolve issues >>> like where the config files are located etc) >>> >>> >>> On Wed, Mar 13, 2013 at 7:29 PM, Cyril Bogus <[EMAIL PROTECTED]>wrote: >>> >>>> Thanks for the reply. >>>> >>>> I am running on Linux. The problem IS the config file but since I >>>> couldn't figure out where, I made two copies of hadoop. One where it is >>>> looking for it, and where other nodes will be looking for it. >>>> >>>> In my config file, everything is I have set hadoop in /home/hadoop but >>>> it is looking for it in /home/owner/hadoop (for some processes at least) >>>> and that was the problem. >>>> >>>> >>>> On Tue, Mar 12, 2013 at 11:06 AM, Jean-Marc Spaggiari < >>>> [EMAIL PROTECTED]> wrote: >>>> >>>>> Hi Cyril, >>>>> >>>>> Are you running in Cygwin? Or Linux? Also, you might want to share >>>>> your configuration files if you want someone to take a look. >>>>> >>>>> JM >>>>> >>>>> 2013/3/8 Cyril Bogus <[EMAIL PROTECTED]>: >>>>> > I am trying to have a two node cluster. >>>>> > Hadoop 1.0.4 >>>>> > >>>>> > the master is under the account A >>>>> > the slave is under the account B >>>>> > >>>>> > I run into an exception when I try to start the cluster. >>>>> > Here is my output >>>>> > >>>>> > A@owner-5:~$ /home/hadoop/bin/start-all.sh >>>>> > Warning: $HADOOP_HOME is deprecated. >>>>> > >>>>> > namenode running as process 3020. Stop it first. >>>>> > A@master: starting datanode, logging to >>>>> > /home/hadoop/libexec/../logs/hadoop-A-datanode-owner-5.out >>>>> > B@slave: Warning: $HADOOP_HOME is deprecated. >>>>> > B@slave: >>>>> > B@slave: starting datanode, logging to >>>>> > /home/owner/hadoop//logs/hadoop-owner-datanode-owner-7.out >>>>> > B@slave: Exception in thread "main" java.lang.NoClassDefFoundError: >>>>> > org/apache/hadoop/util/PlatformName >>>>> > B@slave: Caused by: java.lang.ClassNotFoundException: >>>>> > org.apache.hadoop.util.PlatformName >>>>> > B@slave: at >>>>> java.net.URLClassLoader$1.run(URLClassLoader.java:202) >>>>> > B@slave: at java.security.AccessController.doPrivileged(Native >>>>> Method) >>>>> > B@slave: at >>>>> java.net.URLClassLoader.findClass(URLClassLoader.java:190) >>>>> > B@slave: at >>>>> java.lang.ClassLoader.loadClass(ClassLoader.java:306) >>>>> > B@slave: at >>>>> > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) >>>>> > B@slave: at >>>>> java.lang.ClassLoader.loadClass(ClassLoader.java:247) >>>>> > B@slave: Could not find the main class: >>>>> org.apache.hadoop.util.PlatformName. >>>>> > Program will exit. >>>>> > B@slave: Exception in thread "main" java.lang.NoClassDefFoundError: >>>>> > org/apache/hadoop/hdfs/server/datanode/DataNode >>>>> > A@master: secondarynamenode running as process 3352. Stop it first. >>>>> > starting jobtracker, logging to >>>>> > /home/hadoop/libexec/../logs/hadoop-A-jobtracker-owner-5.out >>>>> > B@slave: Warning: $HADOOP_HOME is deprecated. >>>>> > B@slave: >>>>> > B@slave: starting tasktracker, logging to >>>>> > /home/owner/hadoop//logs/hadoop-owner-tasktracker-owner-7.out >>>>> > A@master: starting tasktracker, logging to
|
|