Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # dev >> Building Hadoop from source code


+
Mohammad Mustaqeem 2013-04-08, 15:06
+
Mohammad Mustaqeem 2013-04-08, 15:08
+
Steve Loughran 2013-04-08, 18:39
+
Mohammad Mustaqeem 2013-04-09, 08:08
+
Ling Kun 2013-04-09, 08:28
+
maisnam ns 2013-04-09, 08:38
+
Mohammad Mustaqeem 2013-04-09, 09:02
+
Mohammad Mustaqeem 2013-04-09, 11:24
+
maisnam ns 2013-04-09, 11:45
+
maisnam ns 2013-04-09, 11:52
+
Mohammad Mustaqeem 2013-04-09, 11:58
+
Ling Kun 2013-04-09, 12:07
+
Mohammad Mustaqeem 2013-04-09, 12:25
+
Ling Kun 2013-04-09, 12:59
+
Mohammad Mustaqeem 2013-04-09, 13:15
+
Harsh J 2013-04-09, 13:48
+
Amir Sanjar 2013-04-09, 13:30
+
Mohammad Mustaqeem 2013-04-09, 13:36
Copy link to this message
-
Re: Building Hadoop from source code

first setup the build environment. I use a script to setup following
properties, you need to modify it:
export M2_HOME=~/development/apache-maven-3.0.4
export M2=$M2_HOME/bin
export ANT_HOME=/opt/apache-ant-1.8.4
export PATH=$M2:$ANT_HOME/bin:$PATH:
export GREP_OPTIONS='--color=auto'
export HADOOP_HOME=~/development/hadoop-1.1.2
export HADOOP_LOG_DIR=$HADOOP_HOME/logs
export ECLIPSE_HOME=~/development/eclipse
export FORREST_HOME=~/development/apache-forrest-0.8
export PATH=$PATH:$FORREST_HOME/bin
export JAVA_HOME=~/development/ibm-java-x86_64-60
export PATH$JAVA_HOME/bin:$PATH:$M2:$ANT_HOME/bin:~/development/protobuf-2.4.1/src:/usr/lib64

export CLASSPATH=$ANT_HOME/lib:$CLASSPATH:$JAVA_HOME/jre/lib:$ANT_HOME/lib:
export LD_LIBRARY_PATH$HADOOP_HOME/build/native/Linux-amd64-64/lib:$ANT_HOME/lib:/usr/lib64
umask 0022

then build by:
ant clean package -Djava5.home=/usr/java/jdk1.5.0_22
-Dforrest.home=/home/sanjar/development/apache-forrest-0.8
-Dcompile.native=true

Best Regards
Amir Sanjar

System Management Architect
PowerLinux Open Source Hadoop development lead
IBM Senior Software Engineer
Phone# 512-286-8393
Fax#      512-838-8858

From: Mohammad Mustaqeem <[EMAIL PROTECTED]>
To: common-dev <[EMAIL PROTECTED]>,
Date: 04/09/2013 08:40 AM
Subject: Re: Building Hadoop from source code

@Amir, how to compile this??
The process will be same as using "mvn package -Pdist -Dtar -DskipTests" or
something else??
On Tue, Apr 9, 2013 at 7:00 PM, Amir Sanjar <[EMAIL PROTECTED]> wrote:

> try hadoop 1.1.2..
>
> Best Regards
> Amir Sanjar
>
> System Management Architect
> PowerLinux Open Source Hadoop development lead
> IBM Senior Software Engineer
> Phone# 512-286-8393
> Fax#      512-838-8858
>
>
>
> [image: Inactive hide details for Mohammad Mustaqeem ---04/09/2013
> 08:18:02 AM---@Ling, i am very new. I was about to ask you the
same.]Mohammad
> Mustaqeem ---04/09/2013 08:18:02 AM---@Ling, i am very new. I was about
> to ask you the same.
>
> From: Mohammad Mustaqeem <[EMAIL PROTECTED]>
> To: common-dev <[EMAIL PROTECTED]>,
> Date: 04/09/2013 08:18 AM
> Subject: Re: Building Hadoop from source code
> ------------------------------
>
>
>
> @Ling, i am very new.
> I was about to ask you the same.
> Which version is easy to handle?
> I want to change something in HDFS.
> Please, suggest.
>
>
> On Tue, Apr 9, 2013 at 6:29 PM, Ling Kun <[EMAIL PROTECTED]> wrote:
>
> > @Mohammad
> >
> >    Sorry, since the internet connection rate I am using is tooooooo
low,
> It
> > takes me a looooong time to download the tar file you showed above.
> >
> >    however, here is some of my opinion based on my local develop env,
> and i
> > am also a fresh hadooper here.
> >
> >    1. what you have download from the trunk is Hadoop 3.0, which you
can
> > see from the tar.gz filename.  While the link above is about Hadoop
1.0,
> > and 2.0. So it seems hadoop developers have adjust the distributed tar
> > file.
> >
> >    2. When looking into the bin/hadoop script in both tar.gz file. I
> think
> > that you will find out that most of them are the same. This file will
> make
> > a MR job start.
> >
> >    3. What version of hadoop do you plan to work on? 2.0 or 3.0?
> >
> >   Ling Kun
> >
> >
> > On Tue, Apr 9, 2013 at 8:25 PM, Mohammad Mustaqeem
> > <[EMAIL PROTECTED]>wrote:
> >
> > > @Ling, Is it the same file as we download from
> > > http://apache.techartifact.com/mirror/hadoop/common/ ??
> > > But the tree structure of the build tar file is not same as
downloaded
> > from
> > > the site.
> > >
> > > @Ling plz guide me. I have been stuck at this state from 1 month .
> > > I didn't find link that explain in detail. The material that I found
is
> > > very old that uses ant.
> > > I want to just build hadoop from source code.
> > > Plz, tell
> > >
> > >
> > > On Tue, Apr 9, 2013 at 5:37 PM, Ling Kun <[EMAIL PROTECTED]> wrote:
> > >
> > > > @Mohammad:
> > > >   1. have you successfully run the command Steve suggested?  "mvn
"*tar*"
Mohammad
<[EMAIL PROTECTED]>
hadoop
will
so
file??
can
faster
*With regards ---*
*Mohammad Mustaqeem*,
M.Tech (CSE)
MNNIT Allahabad
9026604270
+
Mohammad Mustaqeem 2013-04-08, 15:11
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB