Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # dev >> Supporting cross-project Jenkins builds


Copy link to this message
-
Re: Supporting cross-project Jenkins builds
Giri,

I agree that running ALL tests all time takes a lot of time
(personally I'd prefer we do this at the penalty of longer runs).

Still we have a problem to solve, we need to find a solution on
test-patch working for ALL maven modules, currently changes outside of
common/hdfs/mapred or cross-projects test-patch does not work.

So, how about the following approach:

* All patches must be at trunk/ level
* All patches do a full clean TARBALL creation without running testcases
* From the patch file we find out the maven modules and for those
modules we do javac-warns/javadoc-warns/findbugs/testcases

This would speed up test-patch runs and together with a nightly
jenkins jobs running ALL testcases would give a complete coverage.

Does this seem reasonable?

Thxs.

Alejandro

On Tue, Apr 17, 2012 at 3:31 PM, Tom White <[EMAIL PROTECTED]> wrote:
> Giri,
>
> I think Aaron was talking about not running all test cases for changes
> to any project (e.g. HDFS and MapReduce). My proposal was to run all
> the tests for any Common change. An HDFS change would only run HDFS
> tests, and any MapReduce change would only run MapReduce tests.
>
> Another thing I didn't mention was that currently Jenkins doesn't run
> tests or apply patches for any changes in hadoop-tools, which would be
> fixed by the change I'm suggesting.
>
> Tom
>
> On Tue, Apr 17, 2012 at 3:17 PM, Giridharan Kesavan
> <[EMAIL PROTECTED]> wrote:
>> I agree with Aaron. Its going increase the test patch build timings
>> significantly which may not be very helpful
>>
>> Im  -1 on this.
>>
>> -Giri
>>
>>
>>
>> On Mon, Apr 16, 2012 at 2:22 PM, Aaron T. Myers <[EMAIL PROTECTED]> wrote:
>>> On Mon, Apr 16, 2012 at 2:14 PM, Alejandro Abdelnur <[EMAIL PROTECTED]>wrote:
>>>
>>>> * all testcases should always be run (else a change in hdfs could
>>>> affect yarn/tools but not be detected, or one in yarn affect tools)
>>>>
>>>
>>> I'm -0 on this suggestion. Yes, it's a nice benefit to check all of the
>>> dependent Hadoop sub-projects for every patch, but it will also
>>> dramatically increase the time test-patch takes to run for any given patch.
>>> In my experience, the vast majority of patches stand little chance of
>>> breaking the dependent sub-projects, making this largely unnecessary and
>>> thus a waste of time and Jenkins build slave resources.
>>>
>>> --
>>> Aaron T. Myers
>>> Software Engineer, Cloudera

--
Alejandro
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB