Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # dev - Supporting cross-project Jenkins builds


Copy link to this message
-
Re: Supporting cross-project Jenkins builds
Alejandro Abdelnur 2012-04-17, 23:52
Giri,

I agree that running ALL tests all time takes a lot of time
(personally I'd prefer we do this at the penalty of longer runs).

Still we have a problem to solve, we need to find a solution on
test-patch working for ALL maven modules, currently changes outside of
common/hdfs/mapred or cross-projects test-patch does not work.

So, how about the following approach:

* All patches must be at trunk/ level
* All patches do a full clean TARBALL creation without running testcases
* From the patch file we find out the maven modules and for those
modules we do javac-warns/javadoc-warns/findbugs/testcases

This would speed up test-patch runs and together with a nightly
jenkins jobs running ALL testcases would give a complete coverage.

Does this seem reasonable?

Thxs.

Alejandro

On Tue, Apr 17, 2012 at 3:31 PM, Tom White <[EMAIL PROTECTED]> wrote:
> Giri,
>
> I think Aaron was talking about not running all test cases for changes
> to any project (e.g. HDFS and MapReduce). My proposal was to run all
> the tests for any Common change. An HDFS change would only run HDFS
> tests, and any MapReduce change would only run MapReduce tests.
>
> Another thing I didn't mention was that currently Jenkins doesn't run
> tests or apply patches for any changes in hadoop-tools, which would be
> fixed by the change I'm suggesting.
>
> Tom
>
> On Tue, Apr 17, 2012 at 3:17 PM, Giridharan Kesavan
> <[EMAIL PROTECTED]> wrote:
>> I agree with Aaron. Its going increase the test patch build timings
>> significantly which may not be very helpful
>>
>> Im  -1 on this.
>>
>> -Giri
>>
>>
>>
>> On Mon, Apr 16, 2012 at 2:22 PM, Aaron T. Myers <[EMAIL PROTECTED]> wrote:
>>> On Mon, Apr 16, 2012 at 2:14 PM, Alejandro Abdelnur <[EMAIL PROTECTED]>wrote:
>>>
>>>> * all testcases should always be run (else a change in hdfs could
>>>> affect yarn/tools but not be detected, or one in yarn affect tools)
>>>>
>>>
>>> I'm -0 on this suggestion. Yes, it's a nice benefit to check all of the
>>> dependent Hadoop sub-projects for every patch, but it will also
>>> dramatically increase the time test-patch takes to run for any given patch.
>>> In my experience, the vast majority of patches stand little chance of
>>> breaking the dependent sub-projects, making this largely unnecessary and
>>> thus a waste of time and Jenkins build slave resources.
>>>
>>> --
>>> Aaron T. Myers
>>> Software Engineer, Cloudera

--
Alejandro