Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # dev - how to set org.apache.hadoop classpath?


Copy link to this message
-
RE: how to set org.apache.hadoop classpath?
Yoonmin Nam 2014-01-28, 04:49
Hello, Harsh!

I am Yoonmin from South Korea.

I have a question for you about hadoop classpath.
My hadoop home folder is same in all nodes in my cluster :
/home/hadoop/hadoop-1.2.1

Then, I compiled my own modified version of hadoop, so I have a file named
"hadoop-core-1.2.2.SNAPSHOT.jar"

Using scp, I distributed that file to all of my nodes at
/home/hadoop/hadoop-1.2.1 and /home/hadoop/hadoop-1.2.1/build.
Then, I restart dfs and mapred. But changed things are not effected when I
submitted new job into my cluster.

I think it was strange. So, I delete hadoop-core-1.2.2.SNAPSHOP.jar from two
folders as I mentioned.
However, the dfs and mapred are started successfully!

What's going on in my hadoop?

I think it was strongly related to the classpath, so I am asking you what
was wrong?

Thank you!

Best Regards
Yoonmin

-----Original Message-----
From: Harsh J [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, January 28, 2014 7:20 AM
To: [EMAIL PROTECTED]
Subject: Re: how to set org.apache.hadoop classpath?

The imports referenced in your error come from the hadoop-common jar, and
won't be present in the hadoop-mapreduce-client-core jar.

The most ideal way to set your compile classpath would be to rely on the
"hadoop classpath" command. Provided $HADOOP_PREFIX/bin/ is on your $PATH,
do the below:

export CLASSPATH=$(hadoop classpath)
javac WordCount.java

On Mon, Jan 27, 2014 at 1:43 PM, EdwardKing <[EMAIL PROTECTED]> wrote:
> I use hadoop-2.2.0 under CentOS-5.8,then I set JAVA_HOME ,HADOOP_HOME
> and CLASSPATH
>
> [hadoop@master ~]$ echo $JAVA_HOME
> /home/software/jdk1.7.0_02
> [hadoop@master ~]$ echo $HADOOP_HOME
> /home/software/hadoop-2.2.0
> [hadoop@master mapreduce]$ echo $CLASSPATH
.:/home/software/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client
-core-2.2.0.jar:
>
> I write a java file,like follows:
>
> import org.apache.hadoop.conf.Configuration;
> import org.apache.hadoop.fs.Path;
> import org.apache.hadoop.io.IntWritable; import
> org.apache.hadoop.io.Text; public class WordCount1{
>   ...........
> }
>
> Then I compile this java file,but it raise following errors,like follows:
>
> [hadoop@master ~]$ javac WordCount1.java
> WordCount1.java:2: error: package org.apache.hadoop.conf does not
> exist import org.apache.hadoop.conf.Configuration;
>                              ^
> WordCount1.java:3: error: package org.apache.hadoop.fs does not exist
> import org.apache.hadoop.fs.Path;
>
> I have set hadoop-mapreduce-client-core-2.2.0.jar into CLASSPATH, why it
still raise above error? Where is wrong?
>
> Thanks.
>
> ----------------------------------------------------------------------
> ----------------------------- Confidentiality Notice: The information
> contained in this e-mail and any accompanying attachment(s) is
> intended only for the use of the intended recipient and may be
> confidential and/or privileged of Neusoft Corporation, its
> subsidiaries and/or its affiliates. If any reader of this
> communication is not the intended recipient, unauthorized use,
> forwarding, printing,  storing, disclosure or copying is strictly
> prohibited, and may be unlawful.If you have received this communication in
error,please immediately notify the sender by return e-mail, and delete the
original message and all copies from your system. Thank you.
> ----------------------------------------------------------------------
> -----------------------------

--
Harsh J