Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop, mail # dev - Building Hadoop from source code


+
Mohammad Mustaqeem 2013-04-08, 15:06
+
Mohammad Mustaqeem 2013-04-08, 15:08
+
Steve Loughran 2013-04-08, 18:39
+
Mohammad Mustaqeem 2013-04-09, 08:08
+
Ling Kun 2013-04-09, 08:28
+
maisnam ns 2013-04-09, 08:38
+
Mohammad Mustaqeem 2013-04-09, 09:02
+
Mohammad Mustaqeem 2013-04-09, 11:24
+
maisnam ns 2013-04-09, 11:45
+
maisnam ns 2013-04-09, 11:52
+
Mohammad Mustaqeem 2013-04-09, 11:58
+
Ling Kun 2013-04-09, 12:07
+
Mohammad Mustaqeem 2013-04-09, 12:25
+
Ling Kun 2013-04-09, 12:59
+
Mohammad Mustaqeem 2013-04-09, 13:15
+
Harsh J 2013-04-09, 13:48
+
Amir Sanjar 2013-04-09, 13:30
+
Mohammad Mustaqeem 2013-04-09, 13:36
Copy link to this message
-
Re: Building Hadoop from source code
Amir Sanjar 2013-04-09, 14:12

first setup the build environment. I use a script to setup following
properties, you need to modify it:
export M2_HOME=~/development/apache-maven-3.0.4
export M2=$M2_HOME/bin
export ANT_HOME=/opt/apache-ant-1.8.4
export PATH=$M2:$ANT_HOME/bin:$PATH:
export GREP_OPTIONS='--color=auto'
export HADOOP_HOME=~/development/hadoop-1.1.2
export HADOOP_LOG_DIR=$HADOOP_HOME/logs
export ECLIPSE_HOME=~/development/eclipse
export FORREST_HOME=~/development/apache-forrest-0.8
export PATH=$PATH:$FORREST_HOME/bin
export JAVA_HOME=~/development/ibm-java-x86_64-60
export PATH$JAVA_HOME/bin:$PATH:$M2:$ANT_HOME/bin:~/development/protobuf-2.4.1/src:/usr/lib64

export CLASSPATH=$ANT_HOME/lib:$CLASSPATH:$JAVA_HOME/jre/lib:$ANT_HOME/lib:
export LD_LIBRARY_PATH$HADOOP_HOME/build/native/Linux-amd64-64/lib:$ANT_HOME/lib:/usr/lib64
umask 0022

then build by:
ant clean package -Djava5.home=/usr/java/jdk1.5.0_22
-Dforrest.home=/home/sanjar/development/apache-forrest-0.8
-Dcompile.native=true

Best Regards
Amir Sanjar

System Management Architect
PowerLinux Open Source Hadoop development lead
IBM Senior Software Engineer
Phone# 512-286-8393
Fax#      512-838-8858

From: Mohammad Mustaqeem <[EMAIL PROTECTED]>
To: common-dev <[EMAIL PROTECTED]>,
Date: 04/09/2013 08:40 AM
Subject: Re: Building Hadoop from source code

@Amir, how to compile this??
The process will be same as using "mvn package -Pdist -Dtar -DskipTests" or
something else??
On Tue, Apr 9, 2013 at 7:00 PM, Amir Sanjar <[EMAIL PROTECTED]> wrote:

> try hadoop 1.1.2..
>
> Best Regards
> Amir Sanjar
>
> System Management Architect
> PowerLinux Open Source Hadoop development lead
> IBM Senior Software Engineer
> Phone# 512-286-8393
> Fax#      512-838-8858
>
>
>
> [image: Inactive hide details for Mohammad Mustaqeem ---04/09/2013
> 08:18:02 AM---@Ling, i am very new. I was about to ask you the
same.]Mohammad
> Mustaqeem ---04/09/2013 08:18:02 AM---@Ling, i am very new. I was about
> to ask you the same.
>
> From: Mohammad Mustaqeem <[EMAIL PROTECTED]>
> To: common-dev <[EMAIL PROTECTED]>,
> Date: 04/09/2013 08:18 AM
> Subject: Re: Building Hadoop from source code
> ------------------------------
>
>
>
> @Ling, i am very new.
> I was about to ask you the same.
> Which version is easy to handle?
> I want to change something in HDFS.
> Please, suggest.
>
>
> On Tue, Apr 9, 2013 at 6:29 PM, Ling Kun <[EMAIL PROTECTED]> wrote:
>
> > @Mohammad
> >
> >    Sorry, since the internet connection rate I am using is tooooooo
low,
> It
> > takes me a looooong time to download the tar file you showed above.
> >
> >    however, here is some of my opinion based on my local develop env,
> and i
> > am also a fresh hadooper here.
> >
> >    1. what you have download from the trunk is Hadoop 3.0, which you
can
> > see from the tar.gz filename.  While the link above is about Hadoop
1.0,
> > and 2.0. So it seems hadoop developers have adjust the distributed tar
> > file.
> >
> >    2. When looking into the bin/hadoop script in both tar.gz file. I
> think
> > that you will find out that most of them are the same. This file will
> make
> > a MR job start.
> >
> >    3. What version of hadoop do you plan to work on? 2.0 or 3.0?
> >
> >   Ling Kun
> >
> >
> > On Tue, Apr 9, 2013 at 8:25 PM, Mohammad Mustaqeem
> > <[EMAIL PROTECTED]>wrote:
> >
> > > @Ling, Is it the same file as we download from
> > > http://apache.techartifact.com/mirror/hadoop/common/ ??
> > > But the tree structure of the build tar file is not same as
downloaded
> > from
> > > the site.
> > >
> > > @Ling plz guide me. I have been stuck at this state from 1 month .
> > > I didn't find link that explain in detail. The material that I found
is
> > > very old that uses ant.
> > > I want to just build hadoop from source code.
> > > Plz, tell
> > >
> > >
> > > On Tue, Apr 9, 2013 at 5:37 PM, Ling Kun <[EMAIL PROTECTED]> wrote:
> > >
> > > > @Mohammad:
> > > >   1. have you successfully run the command Steve suggested?  "mvn
"*tar*"
Mohammad
<[EMAIL PROTECTED]>
hadoop
will
so
file??
can
faster
*With regards ---*
*Mohammad Mustaqeem*,
M.Tech (CSE)
MNNIT Allahabad
9026604270
+
Mohammad Mustaqeem 2013-04-08, 15:11