MapReduce, mail # dev - Re: Problem with MRv2 Build

Problem with MRv2 Build

Praveen Sripati 2011-08-04, 15:13

Hi,

I did get the latest code from

http://svn.apache.org/repos/asf/hadoop/common/branches/MR-279/

and ran the below command for mapreduce build

mvn clean install assembly:assembly -DskipTests

to get the following error

[INFO] yarn-api .......................................... FAILURE [2.565s]
[INFO] yarn-common ....................................... SKIPPED
[INFO] yarn-server-common ................................ SKIPPED

[ERROR] Failed to execute goal on project yarn-api: Could not resolve
dependencies for project org.apache.hadoop:yarn-api:jar:1.0-SNAPSHOT: The
following artifacts could not be resolved:
org.apache.hadoop:hadoop-common:jar:0.23.0-SNAPSHOT,
org.apache.hadoop:hadoop-annotations:jar:0.23.0-SNAPSHOT,
org.apache.hadoop:hadoop-common:jar:tests:0.23.0-SNAPSHOT,
org.apache.hadoop:hadoop-hdfs:jar:0.23.0-SNAPSHOT: Could not find artifact
org.apache.hadoop:hadoop-common:jar:0.23.0-SNAPSHOT -> [Help 1]

Can someone please help me out with the build problem? The last time I got
the code from SVN the build was successful, looks like some changes in the
code introduced this problem.

Thanks,
Praveen

Re: Problem with MRv2 Build

Thomas Graves 2011-08-04, 15:19

You have to use common and hdfs from trunk branch now and only mapreduce
comes from MR-279 branch.

Something like this:
svn co http://svn.apache.org/repos/asf/hadoop/common/trunk/
rm -rf mapreduce
svn checkout
http://svn.apache.org/repos/asf/hadoop/common/branches/MR-279/mapreduce

Note that common is now using mvn to build.

Tom
On 8/4/11 10:13 AM, "Praveen Sripati" <[EMAIL PROTECTED]> wrote:

> Hi,
>
> I did get the latest code from
>
> http://svn.apache.org/repos/asf/hadoop/common/branches/MR-279/
>
> and ran the below command for mapreduce build
>
> mvn clean install assembly:assembly -DskipTests
>
> to get the following error
>
> [INFO] yarn-api .......................................... FAILURE [2.565s]
> [INFO] yarn-common ....................................... SKIPPED
> [INFO] yarn-server-common ................................ SKIPPED
>
> [ERROR] Failed to execute goal on project yarn-api: Could not resolve
> dependencies for project org.apache.hadoop:yarn-api:jar:1.0-SNAPSHOT: The
> following artifacts could not be resolved:
> org.apache.hadoop:hadoop-common:jar:0.23.0-SNAPSHOT,
> org.apache.hadoop:hadoop-annotations:jar:0.23.0-SNAPSHOT,
> org.apache.hadoop:hadoop-common:jar:tests:0.23.0-SNAPSHOT,
> org.apache.hadoop:hadoop-hdfs:jar:0.23.0-SNAPSHOT: Could not find artifact
> org.apache.hadoop:hadoop-common:jar:0.23.0-SNAPSHOT -> [Help 1]
>
> Can someone please help me out with the build problem? The last time I got
> the code from SVN the build was successful, looks like some changes in the
> code introduced this problem.
>
> Thanks,
> Praveen

Re: Problem with MRv2 Build

Haoyuan Li 2011-08-10, 03:36

Hi Tom,

I also have trouble in building the current MR-279. How to compile common by
using mvn?

And do you mean that
http://svn.apache.org/repos/asf/hadoop/common/branches/MR-279/mapreduce/INSTALL
doesn't
work anymore?

Thank you.

Best,

Haoyuan
On Thu, Aug 4, 2011 at 8:19 AM, Thomas Graves <[EMAIL PROTECTED]> wrote:

> You have to use common and hdfs from trunk branch now and only mapreduce
> comes from MR-279 branch.
>
> Something like this:
> svn co http://svn.apache.org/repos/asf/hadoop/common/trunk/
> rm -rf mapreduce
> svn checkout
> http://svn.apache.org/repos/asf/hadoop/common/branches/MR-279/mapreduce
>
> Note that common is now using mvn to build.
>
> Tom
>
>
> On 8/4/11 10:13 AM, "Praveen Sripati" <[EMAIL PROTECTED]> wrote:
>
> > Hi,
> >
> > I did get the latest code from
> >
> > http://svn.apache.org/repos/asf/hadoop/common/branches/MR-279/
> >
> > and ran the below command for mapreduce build
> >
> > mvn clean install assembly:assembly -DskipTests
> >
> > to get the following error
> >
> > [INFO] yarn-api .......................................... FAILURE
> [2.565s]
> > [INFO] yarn-common ....................................... SKIPPED
> > [INFO] yarn-server-common ................................ SKIPPED
> >
> > [ERROR] Failed to execute goal on project yarn-api: Could not resolve
> > dependencies for project org.apache.hadoop:yarn-api:jar:1.0-SNAPSHOT: The
> > following artifacts could not be resolved:
> > org.apache.hadoop:hadoop-common:jar:0.23.0-SNAPSHOT,
> > org.apache.hadoop:hadoop-annotations:jar:0.23.0-SNAPSHOT,
> > org.apache.hadoop:hadoop-common:jar:tests:0.23.0-SNAPSHOT,
> > org.apache.hadoop:hadoop-hdfs:jar:0.23.0-SNAPSHOT: Could not find
> artifact
> > org.apache.hadoop:hadoop-common:jar:0.23.0-SNAPSHOT -> [Help 1]
> >
> > Can someone please help me out with the build problem? The last time I
> got
> > the code from SVN the build was successful, looks like some changes in
> the
> > code introduced this problem.
> >
> > Thanks,
> > Praveen
>
>

Re: Problem with MRv2 Build

Praveen Sripati 2011-08-10, 03:58

Haoyuan,

The instructions at the mentioned URL are not updated.

1. Get the code from trunk.
2. Remove the mapreduce folder.
3. Get the mapreduce code from MR-279 branch.
4. Run the build scripts.

echo "Common build ....."
mvn clean install package -Pbintar -DskipTests

echo "HDFS build ....."
cd hdfs
ant veryclean mvn-install -Dresolvers=internal

echo "MapReduce build ....."
cd ../mapreduce
mvn clean install assembly:assembly -DskipTests
ant veryclean jar jar-test -Dresolvers=internal

The hadoop-mapreduce-1.0-SNAPSHOT-all.tar.gz file will be generated in the
mapreduce/target folder.

Rest of the instructions or more or less the same for running the sample
program.

Thanks,
Praveen
On Wed, Aug 10, 2011 at 9:06 AM, Haoyuan Li <[EMAIL PROTECTED]> wrote:

> Hi Tom,
>
> I also have trouble in building the current MR-279. How to compile common
> by using mvn?
>
> And do you mean that
> http://svn.apache.org/repos/asf/hadoop/common/branches/MR-279/mapreduce/INSTALL doesn't
> work anymore?
>
> Thank you.
>
> Best,
>
> Haoyuan
>
>
> On Thu, Aug 4, 2011 at 8:19 AM, Thomas Graves <[EMAIL PROTECTED]>wrote:
>
>> You have to use common and hdfs from trunk branch now and only mapreduce
>> comes from MR-279 branch.
>>
>> Something like this:
>> svn co http://svn.apache.org/repos/asf/hadoop/common/trunk/
>> rm -rf mapreduce
>> svn checkout
>> http://svn.apache.org/repos/asf/hadoop/common/branches/MR-279/mapreduce
>>
>> Note that common is now using mvn to build.
>>
>> Tom
>>
>>
>> On 8/4/11 10:13 AM, "Praveen Sripati" <[EMAIL PROTECTED]> wrote:
>>
>> > Hi,
>> >
>> > I did get the latest code from
>> >
>> > http://svn.apache.org/repos/asf/hadoop/common/branches/MR-279/
>> >
>> > and ran the below command for mapreduce build
>> >
>> > mvn clean install assembly:assembly -DskipTests
>> >
>> > to get the following error
>> >
>> > [INFO] yarn-api .......................................... FAILURE
>> [2.565s]
>> > [INFO] yarn-common ....................................... SKIPPED
>> > [INFO] yarn-server-common ................................ SKIPPED
>> >
>> > [ERROR] Failed to execute goal on project yarn-api: Could not resolve
>> > dependencies for project org.apache.hadoop:yarn-api:jar:1.0-SNAPSHOT:
>> The
>> > following artifacts could not be resolved:
>> > org.apache.hadoop:hadoop-common:jar:0.23.0-SNAPSHOT,
>> > org.apache.hadoop:hadoop-annotations:jar:0.23.0-SNAPSHOT,
>> > org.apache.hadoop:hadoop-common:jar:tests:0.23.0-SNAPSHOT,
>> > org.apache.hadoop:hadoop-hdfs:jar:0.23.0-SNAPSHOT: Could not find
>> artifact
>> > org.apache.hadoop:hadoop-common:jar:0.23.0-SNAPSHOT -> [Help 1]
>> >
>> > Can someone please help me out with the build problem? The last time I
>> got
>> > the code from SVN the build was successful, looks like some changes in
>> the
>> > code introduced this problem.
>> >
>> > Thanks,
>> > Praveen
>>
>>
>

Re: Problem with MRv2 Build

Haoyuan Li 2011-08-10, 05:24

Hi Praveen,

Thank you for the info. It did help. Everything before "mvn clean install
assembly:assembly -DskipTests" worked well, and I got the following errors.
Do you have any clue?

Best,

Haoyuan

main:
   [echo] project.build.directory:
/home/haoyuan/hadoop/trunk/mapreduce/yarn/yarn-server/yarn-server-nodemanager/target
   [copy] Copying 1 file to
/home/haoyuan/hadoop/trunk/mapreduce/yarn/yarn-server/yarn-server-nodemanager/target
   [copy] Copying
/home/haoyuan/hadoop/trunk/mapreduce/yarn/yarn-server/yarn-server-nodemanager/pom.xml
to
/home/haoyuan/hadoop/trunk/mapreduce/yarn/yarn-server/yarn-server-nodemanager/target/saner-pom.xml
[INFO] Executed tasks
[INFO]
[INFO] --- make-maven-plugin:1.0-beta-1:autoreconf (autoreconf) @
yarn-server-nodemanager ---
[DEBUG] org.codehaus.mojo:make-maven-plugin:jar:1.0-beta-1:
[DEBUG] org.codehaus.plexus:plexus-utils:jar:1.5.6:compile
[DEBUG] org.codehaus.mojo:cbuild-utils:jar:1.0-beta-1:compile
[DEBUG]
org.codehaus.plexus:plexus-container-default:jar:1.0-alpha-9-stable-1:compile
[DEBUG] junit:junit:jar:4.4:test (scope managed from compile)
(version managed from 3.8.1)
[DEBUG] classworlds:classworlds:jar:1.1-alpha-2:compile
[DEBUG] org.apache.maven:maven-core:jar:2.1.0:compile
[DEBUG]
org.apache.maven:maven-plugin-parameter-documenter:jar:2.1.0:compile
[DEBUG]
org.apache.maven.reporting:maven-reporting-api:jar:2.1.0:compile
[DEBUG] org.apache.maven.doxia:doxia-sink-api:jar:1.1:compile
[DEBUG] org.apache.maven.doxia:doxia-logging-api:jar:1.1:compile
[DEBUG]
org.apache.maven:maven-repository-metadata:jar:2.1.0:compile
[DEBUG] org.apache.maven:maven-error-diagnostics:jar:2.1.0:compile
[DEBUG] commons-cli:commons-cli:jar:1.0:compile
[DEBUG] org.apache.maven:maven-plugin-descriptor:jar:2.1.0:compile
[DEBUG]
org.codehaus.plexus:plexus-interactivity-api:jar:1.0-alpha-4:compile
[DEBUG] org.apache.maven:maven-monitor:jar:2.1.0:compile
[DEBUG] org.sonatype.plexus:plexus-sec-dispatcher:jar:1.1:compile
[DEBUG] org.sonatype.plexus:plexus-cipher:jar:1.1:compile
[DEBUG] org.apache.maven:maven-model:jar:2.1.0:compile
[DEBUG] org.apache.maven:maven-artifact-manager:jar:2.1.0:compile
[DEBUG]
backport-util-concurrent:backport-util-concurrent:jar:3.1:compile
[DEBUG] org.codehaus.plexus:plexus-interpolation:jar:1.8.1:compile
[DEBUG] org.apache.maven.shared:file-management:jar:1.2.1:compile
[DEBUG] org.apache.maven.shared:maven-shared-io:jar:1.1:compile
[DEBUG]
org.apache.maven.wagon:wagon-provider-api:jar:1.0-beta-5:compile (version
managed from 1.0-alpha-6)
[DEBUG] org.apache.maven:maven-plugin-api:jar:2.1.0:compile
[DEBUG] org.apache.maven:maven-artifact:jar:2.1.0:compile
[DEBUG] org.apache.maven:maven-project:jar:2.1.0:compile
[DEBUG] org.apache.maven:maven-settings:jar:2.1.0:compile
[DEBUG] org.apache.maven:maven-profile:jar:2.1.0:compile
[DEBUG] org.apache.maven:maven-plugin-registry:jar:2.1.0:compile
[DEBUG] Created new class realm
plugin>org.codehaus.mojo:make-maven-plugin:1.0-beta-1
[DEBUG] Importing foreign packages into class realm
plugin>org.codehaus.mojo:make-maven-plugin:1.0-beta-1
[DEBUG] Imported: < maven.api
[DEBUG] Populating class realm
plugin>org.codehaus.mojo:make-maven-plugin:1.0-beta-1
[DEBUG] Included: org.codehaus.mojo:make-maven-plugin:jar:1.0-beta-1
[DEBUG] Included: org.codehaus.plexus:plexus-utils:jar:1.5.6
[DEBUG] Included: org.codehaus.mojo:cbuild-utils:jar:1.0-beta-1
[DEBUG] Included: org.apache.maven.reporting:maven-reporting-api:jar:2.1.0
[DEBUG] Included: org.apache.maven.doxia:doxia-sink-api:jar:1.1
[DEBUG] Included: org.apache.maven.doxia:doxia-logging-api:jar:1.1
[DEBUG] Included: commons-cli:commons-cli:jar:1.0
[DEBUG] Included:
org.codehaus.plexus:plexus-interactivity-api:jar:1.0-alpha-4
[DEBUG] Included: org.sonatype.plexus:plexus-sec-dispatcher:jar:1.1
[DEBUG] Included: org.sonatype.plexus:plexus-cipher:jar:1.1
[DEBUG] Included:
backport-util-concurrent:backport-util-concurrent:jar:3.1
[DEBUG] Included: org.codehaus.plexus:plexus-interpolation:jar:1.8.1
[DEBUG] Included: org.apache.maven.shared:file-management:jar:1.2.1
[DEBUG] Included: org.apache.maven.shared:maven-shared-io:jar:1.1
[DEBUG] Excluded:
org.codehaus.plexus:plexus-container-default:jar:1.0-alpha-9-stable-1
[DEBUG] Excluded: junit:junit:jar:4.4
[DEBUG] Excluded: classworlds:classworlds:jar:1.1-alpha-2
[DEBUG] Excluded: org.apache.maven:maven-core:jar:2.1.0
[DEBUG] Excluded:
org.apache.maven:maven-plugin-parameter-documenter:jar:2.1.0
[DEBUG] Excluded: org.apache.maven:maven-repository-metadata:jar:2.1.0
[DEBUG] Excluded: org.apache.maven:maven-error-diagnostics:jar:2.1.0
[DEBUG] Excluded: org.apache.maven:maven-plugin-descriptor:jar:2.1.0
[DEBUG] Excluded: org.apache.maven:maven-monitor:jar:2.1.0
[DEBUG] Excluded: org.apache.maven:maven-model:jar:2.1.0
[DEBUG] Excluded: org.apache.maven:maven-artifact-manager:jar:2.1.0
[DEBUG] Excluded: org.apache.maven.wagon:wagon-provider-api:jar:1.0-beta-5
[DEBUG] Excluded: org.apache.maven:maven-plugin-api:jar:2.1.0
[DEBUG] Excluded: org.apache.maven:maven-artifact:jar:2.1.0
[DEBUG] Excluded: org.apache.maven:maven-project:jar:2.1.0
[DEBUG] Excluded: org.apache.maven:maven-settings:jar:2.1.0
[DEBUG] Excluded: org.apache.maven:maven-profile:jar:2.1.0
[DEBUG] Excluded: org.apache.maven:maven-plugin-registry:jar:2.1.0
[DEBUG] Configuring mojo
org.codehaus.mojo:make-maven-plugin:1.0-beta-1:autoreconf from plugin realm
ClassRealm[plugin>org.codehaus.mojo:make-maven-plugin:1.0-beta-1, parent:
sun.misc.Launcher$AppClassLoader@1a45a877]
[DEBUG] Configuring mojo
'org.codehaus.mojo:make-maven-plugin:1.0-beta-1:autoreconf' with basic
configurator -->
[DEBUG] (f) arguments = [-i]
[DEBUG] (f) workDir /home/haoyuan/hadoop/trunk/mapreduce/yarn/yarn-server/yarn-server-nodemanager/src/main/c/contain

Re: Problem with MRv2 Build

Arun C Murthy 2011-08-10, 05:28

Looks like you don't have autotool chain required to build the native code for launching containers.

It isn't critical - you can run with the java based DefaultContainerExecutor.

Just pass -P-cbuild to maven to skip building the native code.

Arun

On Aug 9, 2011, at 10:24 PM, Haoyuan Li wrote:

> Hi Praveen,
>
> Thank you for the info. It did help. Everything before "mvn clean install
> assembly:assembly -DskipTests" worked well, and I got the following errors.
> Do you have any clue?
>
> Best,
>
> Haoyuan
>
> main:
> [echo] project.build.directory:
> /home/haoyuan/hadoop/trunk/mapreduce/yarn/yarn-server/yarn-server-nodemanager/target
> [copy] Copying 1 file to
> /home/haoyuan/hadoop/trunk/mapreduce/yarn/yarn-server/yarn-server-nodemanager/target
> [copy] Copying
> /home/haoyuan/hadoop/trunk/mapreduce/yarn/yarn-server/yarn-server-nodemanager/pom.xml
> to
> /home/haoyuan/hadoop/trunk/mapreduce/yarn/yarn-server/yarn-server-nodemanager/target/saner-pom.xml
> [INFO] Executed tasks
> [INFO]
> [INFO] --- make-maven-plugin:1.0-beta-1:autoreconf (autoreconf) @
> yarn-server-nodemanager ---
> [DEBUG] org.codehaus.mojo:make-maven-plugin:jar:1.0-beta-1:
> [DEBUG] org.codehaus.plexus:plexus-utils:jar:1.5.6:compile
> [DEBUG] org.codehaus.mojo:cbuild-utils:jar:1.0-beta-1:compile
> [DEBUG]
> org.codehaus.plexus:plexus-container-default:jar:1.0-alpha-9-stable-1:compile
> [DEBUG] junit:junit:jar:4.4:test (scope managed from compile)
> (version managed from 3.8.1)
> [DEBUG] classworlds:classworlds:jar:1.1-alpha-2:compile
> [DEBUG] org.apache.maven:maven-core:jar:2.1.0:compile
> [DEBUG]
> org.apache.maven:maven-plugin-parameter-documenter:jar:2.1.0:compile
> [DEBUG]
> org.apache.maven.reporting:maven-reporting-api:jar:2.1.0:compile
> [DEBUG] org.apache.maven.doxia:doxia-sink-api:jar:1.1:compile
> [DEBUG] org.apache.maven.doxia:doxia-logging-api:jar:1.1:compile
> [DEBUG]
> org.apache.maven:maven-repository-metadata:jar:2.1.0:compile
> [DEBUG] org.apache.maven:maven-error-diagnostics:jar:2.1.0:compile
> [DEBUG] commons-cli:commons-cli:jar:1.0:compile
> [DEBUG] org.apache.maven:maven-plugin-descriptor:jar:2.1.0:compile
> [DEBUG]
> org.codehaus.plexus:plexus-interactivity-api:jar:1.0-alpha-4:compile
> [DEBUG] org.apache.maven:maven-monitor:jar:2.1.0:compile
> [DEBUG] org.sonatype.plexus:plexus-sec-dispatcher:jar:1.1:compile
> [DEBUG] org.sonatype.plexus:plexus-cipher:jar:1.1:compile
> [DEBUG] org.apache.maven:maven-model:jar:2.1.0:compile
> [DEBUG] org.apache.maven:maven-artifact-manager:jar:2.1.0:compile
> [DEBUG]
> backport-util-concurrent:backport-util-concurrent:jar:3.1:compile
> [DEBUG] org.codehaus.plexus:plexus-interpolation:jar:1.8.1:compile
> [DEBUG] org.apache.maven.shared:file-management:jar:1.2.1:compile
> [DEBUG] org.apache.maven.shared:maven-shared-io:jar:1.1:compile
> [DEBUG]
> org.apache.maven.wagon:wagon-provider-api:jar:1.0-beta-5:compile (version
> managed from 1.0-alpha-6)
> [DEBUG] org.apache.maven:maven-plugin-api:jar:2.1.0:compile
> [DEBUG] org.apache.maven:maven-artifact:jar:2.1.0:compile
> [DEBUG] org.apache.maven:maven-project:jar:2.1.0:compile
> [DEBUG] org.apache.maven:maven-settings:jar:2.1.0:compile
> [DEBUG] org.apache.maven:maven-profile:jar:2.1.0:compile
> [DEBUG] org.apache.maven:maven-plugin-registry:jar:2.1.0:compile
> [DEBUG] Created new class realm
> plugin>org.codehaus.mojo:make-maven-plugin:1.0-beta-1
> [DEBUG] Importing foreign packages into class realm
> plugin>org.codehaus.mojo:make-maven-plugin:1.0-beta-1
> [DEBUG] Imported: < maven.api
> [DEBUG] Populating class realm
> plugin>org.codehaus.mojo:make-maven-plugin:1.0-beta-1
> [DEBUG] Included: org.codehaus.mojo:make-maven-plugin:jar:1.0-beta-1
> [DEBUG] Included: org.codehaus.plexus:plexus-utils:jar:1.5.6
> [DEBUG] Included: org.codehaus.mojo:cbuild-utils:jar:1.0-beta-1

Re: Problem with MRv2 Build

Praveen Sripati 2011-08-10, 05:55

Haoyuan,

I am using Ubuntu 11.04, which doesn't come with autoconf/autoreconf. So, I
had to install autoconf (installed autoconf and autoreconf) to get rid of
the above error.

Thanks,
Praveen

On Wed, Aug 10, 2011 at 10:58 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:

> Looks like you don't have autotool chain required to build the native code
> for launching containers.
>
> It isn't critical - you can run with the java based
> DefaultContainerExecutor.
>
> Just pass -P-cbuild to maven to skip building the native code.
>
> Arun
>
> On Aug 9, 2011, at 10:24 PM, Haoyuan Li wrote:
>
> > Hi Praveen,
> >
> > Thank you for the info. It did help. Everything before "mvn clean install
> > assembly:assembly -DskipTests" worked well, and I got the following
> errors.
> > Do you have any clue?
> >
> > Best,
> >
> > Haoyuan
> >
> > main:
> > [echo] project.build.directory:
> >
> /home/haoyuan/hadoop/trunk/mapreduce/yarn/yarn-server/yarn-server-nodemanager/target
> > [copy] Copying 1 file to
> >
> /home/haoyuan/hadoop/trunk/mapreduce/yarn/yarn-server/yarn-server-nodemanager/target
> > [copy] Copying
> >
> /home/haoyuan/hadoop/trunk/mapreduce/yarn/yarn-server/yarn-server-nodemanager/pom.xml
> > to
> >
> /home/haoyuan/hadoop/trunk/mapreduce/yarn/yarn-server/yarn-server-nodemanager/target/saner-pom.xml
> > [INFO] Executed tasks
> > [INFO]
> > [INFO] --- make-maven-plugin:1.0-beta-1:autoreconf (autoreconf) @
> > yarn-server-nodemanager ---
> > [DEBUG] org.codehaus.mojo:make-maven-plugin:jar:1.0-beta-1:
> > [DEBUG] org.codehaus.plexus:plexus-utils:jar:1.5.6:compile
> > [DEBUG] org.codehaus.mojo:cbuild-utils:jar:1.0-beta-1:compile
> > [DEBUG]
> >
> org.codehaus.plexus:plexus-container-default:jar:1.0-alpha-9-stable-1:compile
> > [DEBUG] junit:junit:jar:4.4:test (scope managed from compile)
> > (version managed from 3.8.1)
> > [DEBUG] classworlds:classworlds:jar:1.1-alpha-2:compile
> > [DEBUG] org.apache.maven:maven-core:jar:2.1.0:compile
> > [DEBUG]
> > org.apache.maven:maven-plugin-parameter-documenter:jar:2.1.0:compile
> > [DEBUG]
> > org.apache.maven.reporting:maven-reporting-api:jar:2.1.0:compile
> > [DEBUG] org.apache.maven.doxia:doxia-sink-api:jar:1.1:compile
> > [DEBUG]
> org.apache.maven.doxia:doxia-logging-api:jar:1.1:compile
> > [DEBUG]
> > org.apache.maven:maven-repository-metadata:jar:2.1.0:compile
> > [DEBUG]
> org.apache.maven:maven-error-diagnostics:jar:2.1.0:compile
> > [DEBUG] commons-cli:commons-cli:jar:1.0:compile
> > [DEBUG]
> org.apache.maven:maven-plugin-descriptor:jar:2.1.0:compile
> > [DEBUG]
> > org.codehaus.plexus:plexus-interactivity-api:jar:1.0-alpha-4:compile
> > [DEBUG] org.apache.maven:maven-monitor:jar:2.1.0:compile
> > [DEBUG]
> org.sonatype.plexus:plexus-sec-dispatcher:jar:1.1:compile
> > [DEBUG] org.sonatype.plexus:plexus-cipher:jar:1.1:compile
> > [DEBUG] org.apache.maven:maven-model:jar:2.1.0:compile
> > [DEBUG] org.apache.maven:maven-artifact-manager:jar:2.1.0:compile
> > [DEBUG]
> > backport-util-concurrent:backport-util-concurrent:jar:3.1:compile
> > [DEBUG] org.codehaus.plexus:plexus-interpolation:jar:1.8.1:compile
> > [DEBUG] org.apache.maven.shared:file-management:jar:1.2.1:compile
> > [DEBUG] org.apache.maven.shared:maven-shared-io:jar:1.1:compile
> > [DEBUG]
> > org.apache.maven.wagon:wagon-provider-api:jar:1.0-beta-5:compile (version
> > managed from 1.0-alpha-6)
> > [DEBUG] org.apache.maven:maven-plugin-api:jar:2.1.0:compile
> > [DEBUG] org.apache.maven:maven-artifact:jar:2.1.0:compile
> > [DEBUG] org.apache.maven:maven-project:jar:2.1.0:compile
> > [DEBUG] org.apache.maven:maven-settings:jar:2.1.0:compile
> > [DEBUG] org.apache.maven:maven-profile:jar:2.1.0:compile
> > [DEBUG] org.apache.maven:maven-plugin-registry:jar:2.1.0:compile
> > [DEBUG] Created new class realm
> > plugin>org.codehaus.mojo:make-maven-plugin:1.0-beta-1
> > [DEBUG] Importing foreign packages into class realm

Re: Problem with MRv2 Build

Haoyuan Li 2011-08-10, 06:12

Thank you Praveen and Arun.

Both approaches work!

However, I still got problem in the last step... ant veryclean jar jar-test
-Dresolvers=internal

Any thoughts?

Thanks,

Haoyuan

ivy-resolve-test:
[ivy:resolve] downloading
/home/haoyuan/.m2/repository/org/apache/hadoop/yarn-common/1.0-SNAPSHOT/yarn-common-1.0-SNAPSHOT-tests.jar
...
[ivy:resolve] ... (71kB)
[ivy:resolve] [SUCCESSFUL ]
org.apache.hadoop#yarn-common;1.0-SNAPSHOT!yarn-common.jar (1ms)
[ivy:resolve] downloading
/home/haoyuan/.m2/repository/org/apache/hadoop/hadoop-hdfs-test/0.23.0-SNAPSHOT/hadoop-hdfs-test-0.23.0-SNAPSHOT.jar
...
[ivy:resolve] .................. (1028kB)
[ivy:resolve] [SUCCESSFUL ]
org.apache.hadoop#hadoop-hdfs-test;0.23.0-SNAPSHOT!hadoop-hdfs-test.jar
(2ms)

ivy-retrieve-test:

compile-mapred-test:
   [mkdir] Created dir:
/home/haoyuan/hadoop/trunk/mapreduce/build/test/mapred/classes
   [mkdir] Created dir:
/home/haoyuan/hadoop/trunk/mapreduce/build/test/mapred/testjar
   [mkdir] Created dir:
/home/haoyuan/hadoop/trunk/mapreduce/build/test/mapred/testshell
   [javac] Compiling 322 source files to
/home/haoyuan/hadoop/trunk/mapreduce/build/test/mapred/classes
   [javac]
/home/haoyuan/hadoop/trunk/mapreduce/src/test/mapred/org/apache/hadoop/mapreduce/security/TestBinaryTokenFile.java:150:
getDelegationTokenSecretManager() is not public in
org.apache.hadoop.hdfs.server.namenode.FSNamesystem; cannot be accessed from
outside package
   [javac]
dfsCluster.getNamesystem().getDelegationTokenSecretManager().startThreads();
   [javac] ^
   [javac]
/home/haoyuan/hadoop/trunk/mapreduce/src/test/mapred/org/apache/hadoop/mapreduce/security/TestTokenCache.java:163:
getDelegationTokenSecretManager() is not public in
org.apache.hadoop.hdfs.server.namenode.FSNamesystem; cannot be accessed from
outside package
   [javac]
dfsCluster.getNamesystem().getDelegationTokenSecretManager().startThreads();
   [javac] ^
   [javac]
/home/haoyuan/hadoop/trunk/mapreduce/src/test/mapred/org/apache/hadoop/mapreduce/security/TestTokenCache.java:309:
getDelegationTokenSecretManager() is not public in
org.apache.hadoop.hdfs.server.namenode.FSNamesystem; cannot be accessed from
outside package
   [javac]
dfsCluster.getNamesystem().getDelegationTokenSecretManager();
   [javac] ^
   [javac]
/home/haoyuan/hadoop/trunk/mapreduce/src/test/mapred/org/apache/hadoop/mapreduce/security/TestTokenCacheOldApi.java:189:
getDelegationTokenSecretManager() is not public in
org.apache.hadoop.hdfs.server.namenode.FSNamesystem; cannot be accessed from
outside package
   [javac] .getDelegationTokenSecretManager().startThreads();
   [javac] ^
   [javac] Note: Some input files use or override a deprecated API.
   [javac] Note: Recompile with -Xlint:deprecation for details.
   [javac] 4 errors

BUILD FAILED
/home/haoyuan/hadoop/trunk/mapreduce/build.xml:520: The following error
occurred while executing this line:
/home/haoyuan/hadoop/trunk/mapreduce/build.xml:589: Compile failed; see the
compiler error output for details.

On Tue, Aug 9, 2011 at 10:55 PM, Praveen Sripati
<[EMAIL PROTECTED]>wrote:

> Haoyuan,
>
> I am using Ubuntu 11.04, which doesn't come with autoconf/autoreconf. So, I
> had to install autoconf (installed autoconf and autoreconf) to get rid of
> the above error.
>
> Thanks,
> Praveen
>
> On Wed, Aug 10, 2011 at 10:58 AM, Arun C Murthy <[EMAIL PROTECTED]>
> wrote:
>
> > Looks like you don't have autotool chain required to build the native
> code
> > for launching containers.
> >
> > It isn't critical - you can run with the java based
> > DefaultContainerExecutor.
> >
> > Just pass -P-cbuild to maven to skip building the native code.
> >
> > Arun
> >
> > On Aug 9, 2011, at 10:24 PM, Haoyuan Li wrote:
> >
> > > Hi Praveen,
> > >
> > > Thank you for the info. It did help. Everything before "mvn clean
> install
> > > assembly:assembly -DskipTests" worked well, and I got the following

Re: Problem with MRv2 Build

Haoyuan Li 2011-08-10, 06:42

It turns out "ant veryclean jar -Dresolvers=internal" can compile well. The
error came from jar-test.

Best,

Haoyuan
On Tue, Aug 9, 2011 at 11:12 PM, Haoyuan Li <[EMAIL PROTECTED]> wrote:

> Thank you Praveen and Arun.
>
> Both approaches work!
>
> However, I still got problem in the last step... ant veryclean jar
> jar-test -Dresolvers=internal
>
> Any thoughts?
>
> Thanks,
>
> Haoyuan
>
> ivy-resolve-test:
> [ivy:resolve] downloading
> /home/haoyuan/.m2/repository/org/apache/hadoop/yarn-common/1.0-SNAPSHOT/yarn-common-1.0-SNAPSHOT-tests.jar
> ...
> [ivy:resolve] ... (71kB)
> [ivy:resolve] [SUCCESSFUL ]
> org.apache.hadoop#yarn-common;1.0-SNAPSHOT!yarn-common.jar (1ms)
> [ivy:resolve] downloading
> /home/haoyuan/.m2/repository/org/apache/hadoop/hadoop-hdfs-test/0.23.0-SNAPSHOT/hadoop-hdfs-test-0.23.0-SNAPSHOT.jar
> ...
> [ivy:resolve] .................. (1028kB)
> [ivy:resolve] [SUCCESSFUL ]
> org.apache.hadoop#hadoop-hdfs-test;0.23.0-SNAPSHOT!hadoop-hdfs-test.jar
> (2ms)
>
> ivy-retrieve-test:
>
> compile-mapred-test:
> [mkdir] Created dir:
> /home/haoyuan/hadoop/trunk/mapreduce/build/test/mapred/classes
> [mkdir] Created dir:
> /home/haoyuan/hadoop/trunk/mapreduce/build/test/mapred/testjar
> [mkdir] Created dir:
> /home/haoyuan/hadoop/trunk/mapreduce/build/test/mapred/testshell
> [javac] Compiling 322 source files to
> /home/haoyuan/hadoop/trunk/mapreduce/build/test/mapred/classes
> [javac]
> /home/haoyuan/hadoop/trunk/mapreduce/src/test/mapred/org/apache/hadoop/mapreduce/security/TestBinaryTokenFile.java:150:
> getDelegationTokenSecretManager() is not public in
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem; cannot be accessed from
> outside package
> [javac]
> dfsCluster.getNamesystem().getDelegationTokenSecretManager().startThreads();
> [javac] ^
> [javac]
> /home/haoyuan/hadoop/trunk/mapreduce/src/test/mapred/org/apache/hadoop/mapreduce/security/TestTokenCache.java:163:
> getDelegationTokenSecretManager() is not public in
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem; cannot be accessed from
> outside package
> [javac]
> dfsCluster.getNamesystem().getDelegationTokenSecretManager().startThreads();
> [javac] ^
> [javac]
> /home/haoyuan/hadoop/trunk/mapreduce/src/test/mapred/org/apache/hadoop/mapreduce/security/TestTokenCache.java:309:
> getDelegationTokenSecretManager() is not public in
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem; cannot be accessed from
> outside package
> [javac]
> dfsCluster.getNamesystem().getDelegationTokenSecretManager();
> [javac] ^
> [javac]
> /home/haoyuan/hadoop/trunk/mapreduce/src/test/mapred/org/apache/hadoop/mapreduce/security/TestTokenCacheOldApi.java:189:
> getDelegationTokenSecretManager() is not public in
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem; cannot be accessed from
> outside package
> [javac] .getDelegationTokenSecretManager().startThreads();
> [javac] ^
> [javac] Note: Some input files use or override a deprecated API.
> [javac] Note: Recompile with -Xlint:deprecation for details.
> [javac] 4 errors
>
> BUILD FAILED
> /home/haoyuan/hadoop/trunk/mapreduce/build.xml:520: The following error
> occurred while executing this line:
> /home/haoyuan/hadoop/trunk/mapreduce/build.xml:589: Compile failed; see the
> compiler error output for details.
>
>
>
> On Tue, Aug 9, 2011 at 10:55 PM, Praveen Sripati <[EMAIL PROTECTED]
> > wrote:
>
>> Haoyuan,
>>
>> I am using Ubuntu 11.04, which doesn't come with autoconf/autoreconf. So,
>> I
>> had to install autoconf (installed autoconf and autoreconf) to get rid of
>> the above error.
>>
>> Thanks,
>> Praveen
>>
>> On Wed, Aug 10, 2011 at 10:58 AM, Arun C Murthy <[EMAIL PROTECTED]>
>> wrote:
>>
>> > Looks like you don't have autotool chain required to build the native
>> code
>> > for launching containers.
>> >
>> > It isn't critical - you can run with the java based