Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # general - getting started building Mavenized hadoop common


Copy link to this message
-
getting started building Mavenized hadoop common
Alejandro Abdelnur 2011-08-02, 21:13
With the HADOOP-6671 commit the way of building hadoop common has changed
significantly.

While the wiki explains these changes, and there is a BUILDING.txt
directory, still I guess things will hit many of of you.

Because of this I've put together some brief notes.

Thanks.

Alejandro

----------------------
NEW LAYOUT

After updating the trunk you'll the the following directory changes at top
level

 Removed: common/
 New: hadoop-common/, hadoop-project/, hadoop-annotations/,
hadoop-assemblies/

* hadoop-common/ is the new common/ and its sub-dirs are organized following
Maven standard project layout.
* hadoop-project/ contains Hadoop project root POM, all dependency versions
are defined there
* hadoop-annotations/ contains the Hadoop public/private annotation classes
* hadoop-assemblies/ contains the assembly files that create the
distribution directories layout

----------------------
BUILDING REQUIREMENTS

The only new build requirement is Maven 3 (it must be at least Maven 3).

The environment var FORREST_HOME must be set if building the documentation.

----------------------
FIRST MAVEN BUILD

It must be run from the trunk/ directory.

Run: 'mvn install -DskipTests'

This will install the different submodules
(project/annotations/assemblies/common) into the local Maven cache
(~/.m2/repository).

After this is done, you can build from the hadoop-commons directory.

NOTE: this will not be required once the SNAPSHOTS Maven repo has the
snapshots published.
----------------------
TARGET/ IS THE NEW BUILD/

The new build directory is target/

----------------------
USING AN IDE

Eclipse and IntelliJ will recognize and open the project from the POM file.

Make sure you run a 'mvn test -DskipTests' every time you have a clean
target/ directory as Maven generates code required for testing and sets some
directories under target/

----------------------
BUILDING

Run 'mvn compile'

To compile native code add '-Pnative'

----------------------
RUNNING TESTCASES

Run 'mvn test -Dtest=TESTCASECLASS'

To run multiple testcases separate the testcases name with comma

To run all testcases don't specifiy -'Dtest=...'

NOTE: TESTCASECLASS is just the testcase classname, no package name, no
extension.

----------------------
CREATING THE TAR

Run 'mvn package -Pbintar -DskipTests'

NOTE: The '-Ptar' profile will create the legacy layout, but the Hadoop
scripts will not work with the legacy layout (this has been the case before
 HADOOP-6671)

----------------------
RUNNING THE HADOOP SCRIPTS IN DEVELOPMENT

Run 'mvn package -Pbintar -DskipTests'

The Hadoop scripts can be executed from
hadoop-common/target/hadoop-common-0.23.0-SNAPSHOT-bin/bin/ directory.

----------------------