Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # general - follow up Hadoop mavenization work


Copy link to this message
-
Re: follow up Hadoop mavenization work
Tom White 2011-08-02, 16:16
On Sun, Jul 31, 2011 at 9:57 PM, Tom White <[EMAIL PROTECTED]> wrote:
> HADOOP-6671 has now received two +1s (one from Eric Yang and one from
> me), so I would like to commit it on Tuesday at 16:00 GMT
> (http://s.apache.org/6nx). I'll also update the Jenkins jobs for
> running test-patch and performing the nightly build.

I'm going to go ahead and commit the patch now.

Tom

>
> For developers this change will mean that you need to use Maven to
> build Hadoop Common. The build instructions are listed in the
> BUILDING.txt file in the patch, as well as at
> http://wiki.apache.org/hadoop/HowToContribute and
> http://s.apache.org/wb.
>
> Note that HDFS and MapReduce will still use Ant for building, but
> follow on JIRAs HDFS-2096 and MAPREDUCE-2607 will introduce Maven to
> those builds in the near future. (In terms of staging, it makes sense
> for MAPREDUCE-2607 to go in after MAPREDUCE-279, since the MR2 work
> uses Maven to build its new modules, so the Mavenization of MapReduce
> should build on that work.)
>
> Thanks,
> Tom
>
> On Thu, Jul 28, 2011 at 4:41 PM, Alejandro Abdelnur <[EMAIL PROTECTED]> wrote:
>> Following up with Hadoop Common mavenization (HADOOP-6671) I've just posted
>> a patch for HDFS mavenization (HDFS-2096)
>>
>> The HADOOP-6671 patch integrates all feedback received in the JIRA and, IMO,
>> it is ready for prime time.
>>
>> In order not break HDFS and MAPRED which are still Ant based, there are 2
>> patches HDFS-2196 & MAPREDUCE-2741that make some corrections in the ivy
>> configuration to work correctly with the Hadoop common JAR (build/published
>> by Mavenized build).
>>
>> HDFS-2096 is not 100% ready, some testcases are failing and native code
>> testing is not wired, but everything else (compile, test, package, tar,
>> binary, jdiff, etc is wired).
>>
>> * https://issues.apache.org/jira/browse/HADOOP-6671
>> * https://issues.apache.org/jira/browse/HDFS-2196
>> * https://issues.apache.org/jira/browse/MAPREDUCE-2741
>> * https://issues.apache.org/jira/browse/HDFS-2096
>>
>> I know these are big changes and we'll have some hiccups, but the benefits
>> are big (running testcases is faster, it easily works from IDEs, Maven build
>> system can easily be understood by anybody that knows Maven).
>>
>> Keeping the patches current is time-consuming, because of this, it would be
>> great if we can get in the ones ready (HADOOP-6671, HDFS-2196,
>> MAPREDUCE-2741) so we can focus on the rest of the Mavenization work.
>>
>> Thanks.
>>
>> Alejandro
>>
>