Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # general >> getting started building Mavenized hadoop common


Copy link to this message
-
getting started building Mavenized hadoop common
With the HADOOP-6671 commit the way of building hadoop common has changed
significantly.

While the wiki explains these changes, and there is a BUILDING.txt
directory, still I guess things will hit many of of you.

Because of this I've put together some brief notes.

Thanks.

Alejandro

----------------------
NEW LAYOUT

After updating the trunk you'll the the following directory changes at top
level

 Removed: common/
 New: hadoop-common/, hadoop-project/, hadoop-annotations/,
hadoop-assemblies/

* hadoop-common/ is the new common/ and its sub-dirs are organized following
Maven standard project layout.
* hadoop-project/ contains Hadoop project root POM, all dependency versions
are defined there
* hadoop-annotations/ contains the Hadoop public/private annotation classes
* hadoop-assemblies/ contains the assembly files that create the
distribution directories layout

----------------------
BUILDING REQUIREMENTS

The only new build requirement is Maven 3 (it must be at least Maven 3).

The environment var FORREST_HOME must be set if building the documentation.

----------------------
FIRST MAVEN BUILD

It must be run from the trunk/ directory.

Run: 'mvn install -DskipTests'

This will install the different submodules
(project/annotations/assemblies/common) into the local Maven cache
(~/.m2/repository).

After this is done, you can build from the hadoop-commons directory.

NOTE: this will not be required once the SNAPSHOTS Maven repo has the
snapshots published.
----------------------
TARGET/ IS THE NEW BUILD/

The new build directory is target/

----------------------
USING AN IDE

Eclipse and IntelliJ will recognize and open the project from the POM file.

Make sure you run a 'mvn test -DskipTests' every time you have a clean
target/ directory as Maven generates code required for testing and sets some
directories under target/

----------------------
BUILDING

Run 'mvn compile'

To compile native code add '-Pnative'

----------------------
RUNNING TESTCASES

Run 'mvn test -Dtest=TESTCASECLASS'

To run multiple testcases separate the testcases name with comma

To run all testcases don't specifiy -'Dtest=...'

NOTE: TESTCASECLASS is just the testcase classname, no package name, no
extension.

----------------------
CREATING THE TAR

Run 'mvn package -Pbintar -DskipTests'

NOTE: The '-Ptar' profile will create the legacy layout, but the Hadoop
scripts will not work with the legacy layout (this has been the case before
 HADOOP-6671)

----------------------
RUNNING THE HADOOP SCRIPTS IN DEVELOPMENT

Run 'mvn package -Pbintar -DskipTests'

The Hadoop scripts can be executed from
hadoop-common/target/hadoop-common-0.23.0-SNAPSHOT-bin/bin/ directory.

----------------------
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB