Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # dev >> Building Hadoop from source code


+
Mohammad Mustaqeem 2013-04-08, 15:06
+
Mohammad Mustaqeem 2013-04-08, 15:08
+
Steve Loughran 2013-04-08, 18:39
+
Mohammad Mustaqeem 2013-04-09, 08:08
+
Ling Kun 2013-04-09, 08:28
+
maisnam ns 2013-04-09, 08:38
+
Mohammad Mustaqeem 2013-04-09, 09:02
+
Mohammad Mustaqeem 2013-04-09, 11:24
+
maisnam ns 2013-04-09, 11:45
+
maisnam ns 2013-04-09, 11:52
+
Mohammad Mustaqeem 2013-04-09, 11:58
+
Ling Kun 2013-04-09, 12:07
+
Mohammad Mustaqeem 2013-04-09, 12:25
+
Ling Kun 2013-04-09, 12:59
+
Mohammad Mustaqeem 2013-04-09, 13:15
Copy link to this message
-
Re: Building Hadoop from source code
Mohammad,

Once you've executed a build for a tarball via "mvn package -Pdist
-DskipTests -Dtar", then the prepared composite, distributable tarball
can be found under the directory hadoop-dist/target/.

I'd oppose basing new work on a 1.x branch, as the difference between
that one (~3 years old now) and the trunk/2.x is very very wide and if
you're interested in contributing your work back, it has to go onto
trunk first.
You can settle onto a 2.x or even a 0.23.x release branch for
stability if needed.

On Tue, Apr 9, 2013 at 6:45 PM, Mohammad Mustaqeem
<[EMAIL PROTECTED]> wrote:
> @Ling, i am very new.
> I was about to ask you the same.
> Which version is easy to handle?
> I want to change something in HDFS.
> Please, suggest.
>
>
> On Tue, Apr 9, 2013 at 6:29 PM, Ling Kun <[EMAIL PROTECTED]> wrote:
>
>> @Mohammad
>>
>>    Sorry, since the internet connection rate I am using is tooooooo low, It
>> takes me a looooong time to download the tar file you showed above.
>>
>>    however, here is some of my opinion based on my local develop env, and i
>> am also a fresh hadooper here.
>>
>>    1. what you have download from the trunk is Hadoop 3.0, which you can
>> see from the tar.gz filename.  While the link above is about Hadoop 1.0,
>> and 2.0. So it seems hadoop developers have adjust the distributed tar
>> file.
>>
>>    2. When looking into the bin/hadoop script in both tar.gz file. I think
>> that you will find out that most of them are the same. This file will make
>> a MR job start.
>>
>>    3. What version of hadoop do you plan to work on? 2.0 or 3.0?
>>
>>   Ling Kun
>>
>>
>> On Tue, Apr 9, 2013 at 8:25 PM, Mohammad Mustaqeem
>> <[EMAIL PROTECTED]>wrote:
>>
>> > @Ling, Is it the same file as we download from
>> > http://apache.techartifact.com/mirror/hadoop/common/ ??
>> > But the tree structure of the build tar file is not same as downloaded
>> from
>> > the site.
>> >
>> > @Ling plz guide me. I have been stuck at this state from 1 month .
>> > I didn't find link that explain in detail. The material that I found is
>> > very old that uses ant.
>> > I want to just build hadoop from source code.
>> > Plz, tell
>> >
>> >
>> > On Tue, Apr 9, 2013 at 5:37 PM, Ling Kun <[EMAIL PROTECTED]> wrote:
>> >
>> > > @Mohammad:
>> > >   1. have you successfully run the command Steve suggested?  "mvn
>> package
>> > > -Pdist -Dtar -DskipTests -Dmaven.javadoc.skip=true"
>> > >   2. If yes, and then you can use the command " find . -name "*tar*" "
>> to
>> > > find the tar file, which is in the path
>> > > "./hadoop-dist/target/hadoop-3.0.0-SNAPSHOT.tar.gz"
>> > >
>> > >   3. Sorry, I make a mistake about the tar file name, It is tar.gz
>> > >
>> > >
>> > > @ maisnam ns:
>> > >     I am afraid the version you are using is not the same as Mohammad
>> > used.
>> > >
>> > >
>> > > Ling Kun
>> > >
>> > >
>> > > On Tue, Apr 9, 2013 at 7:58 PM, Mohammad Mustaqeem
>> > > <[EMAIL PROTECTED]>wrote:
>> > >
>> > > > @maisnam, I am using ubuntu and after running svn I get hadoop folder
>> > as
>> > > > given in screenshot.
>> > > > I didn't finf any build folder
>> > > >
>> > > >
>> > > > On Tue, Apr 9, 2013 at 5:22 PM, maisnam ns <[EMAIL PROTECTED]>
>> > wrote:
>> > > >
>> > > >> @@Mohammad Mustaqeem please ignore the earlier mail as the
>> screenshots
>> > > >> where not copied in the mail I have attached a doc file.
>> > > >>
>> > > >> You can follow the steps earlier using the screenshots I have
>> > attached.
>> > > >>
>> > > >>
>> > > >> On Tue, Apr 9, 2013 at 5:15 PM, maisnam ns <[EMAIL PROTECTED]>
>> > > wrote:
>> > > >>
>> > > >>> @Mohammad Mustaqeem please follow the screenshots one by one
>> > > >>>
>> > > >>>
>> > > >>> cd build you will find
>> > > >>>
>> > > >>>
>> > > >>> Now cd hadoop-0.20.3-dev you will find the below screenshot
>> > > >>>
>> > > >>>
>> > > >>> Now the bin contains all the scripts you may need to start hadoop
>> > > >>> 1. Go to that bin directory and format your namenode
>> > > >>> 2. configure you site-xmls

Harsh J
+
Amir Sanjar 2013-04-09, 13:30
+
Mohammad Mustaqeem 2013-04-09, 13:36
+
Amir Sanjar 2013-04-09, 14:12
+
Mohammad Mustaqeem 2013-04-08, 15:11