Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # dev - Problem with MRv2 Build


Copy link to this message
-
Re: Problem with MRv2 Build
Praveen Sripati 2011-08-10, 03:58
Haoyuan,

The instructions at the mentioned URL are not updated.

1. Get the code from trunk.
2. Remove the mapreduce folder.
3. Get the mapreduce code from MR-279 branch.
4. Run the build scripts.

echo "Common build ....."
mvn clean install package -Pbintar -DskipTests

echo "HDFS build ....."
cd hdfs
ant veryclean mvn-install -Dresolvers=internal

echo "MapReduce build ....."
cd ../mapreduce
mvn clean install assembly:assembly -DskipTests
ant veryclean jar jar-test  -Dresolvers=internal

The hadoop-mapreduce-1.0-SNAPSHOT-all.tar.gz file will be generated in the
mapreduce/target folder.

Rest of the instructions or more or less the same for running the sample
program.

Thanks,
Praveen
On Wed, Aug 10, 2011 at 9:06 AM, Haoyuan Li <[EMAIL PROTECTED]> wrote:

> Hi Tom,
>
> I also have trouble in building the current MR-279. How to compile common
> by using mvn?
>
> And do you mean that
> http://svn.apache.org/repos/asf/hadoop/common/branches/MR-279/mapreduce/INSTALL doesn't
> work anymore?
>
> Thank you.
>
> Best,
>
> Haoyuan
>
>
> On Thu, Aug 4, 2011 at 8:19 AM, Thomas Graves <[EMAIL PROTECTED]>wrote:
>
>> You have to use common and hdfs from trunk branch now and only mapreduce
>> comes from MR-279 branch.
>>
>> Something like this:
>> svn co http://svn.apache.org/repos/asf/hadoop/common/trunk/
>> rm -rf mapreduce
>> svn checkout
>> http://svn.apache.org/repos/asf/hadoop/common/branches/MR-279/mapreduce
>>
>> Note that common is now using mvn to build.
>>
>> Tom
>>
>>
>> On 8/4/11 10:13 AM, "Praveen Sripati" <[EMAIL PROTECTED]> wrote:
>>
>> > Hi,
>> >
>> > I did get the latest code from
>> >
>> > http://svn.apache.org/repos/asf/hadoop/common/branches/MR-279/
>> >
>> > and ran the below command for mapreduce build
>> >
>> > mvn clean install assembly:assembly -DskipTests
>> >
>> > to get the following error
>> >
>> > [INFO] yarn-api .......................................... FAILURE
>> [2.565s]
>> > [INFO] yarn-common ....................................... SKIPPED
>> > [INFO] yarn-server-common ................................ SKIPPED
>> >
>> > [ERROR] Failed to execute goal on project yarn-api: Could not resolve
>> > dependencies for project org.apache.hadoop:yarn-api:jar:1.0-SNAPSHOT:
>> The
>> > following artifacts could not be resolved:
>> > org.apache.hadoop:hadoop-common:jar:0.23.0-SNAPSHOT,
>> > org.apache.hadoop:hadoop-annotations:jar:0.23.0-SNAPSHOT,
>> > org.apache.hadoop:hadoop-common:jar:tests:0.23.0-SNAPSHOT,
>> > org.apache.hadoop:hadoop-hdfs:jar:0.23.0-SNAPSHOT: Could not find
>> artifact
>> > org.apache.hadoop:hadoop-common:jar:0.23.0-SNAPSHOT -> [Help 1]
>> >
>> > Can someone please help me out with the build problem? The last time I
>> got
>> > the code from SVN the build was successful, looks like some changes in
>> the
>> > code introduced this problem.
>> >
>> > Thanks,
>> > Praveen
>>
>>
>