Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # dev >> Supporting cross-project Jenkins builds


+
Tom White 2012-04-16, 20:51
+
Aaron T. Myers 2012-04-16, 20:55
+
Alejandro Abdelnur 2012-04-16, 21:14
+
Aaron T. Myers 2012-04-16, 21:22
+
Giridharan Kesavan 2012-04-17, 22:17
+
Tom White 2012-04-17, 22:31
Copy link to this message
-
Re: Supporting cross-project Jenkins builds
Giri,

I agree that running ALL tests all time takes a lot of time
(personally I'd prefer we do this at the penalty of longer runs).

Still we have a problem to solve, we need to find a solution on
test-patch working for ALL maven modules, currently changes outside of
common/hdfs/mapred or cross-projects test-patch does not work.

So, how about the following approach:

* All patches must be at trunk/ level
* All patches do a full clean TARBALL creation without running testcases
* From the patch file we find out the maven modules and for those
modules we do javac-warns/javadoc-warns/findbugs/testcases

This would speed up test-patch runs and together with a nightly
jenkins jobs running ALL testcases would give a complete coverage.

Does this seem reasonable?

Thxs.

Alejandro

On Tue, Apr 17, 2012 at 3:31 PM, Tom White <[EMAIL PROTECTED]> wrote:
> Giri,
>
> I think Aaron was talking about not running all test cases for changes
> to any project (e.g. HDFS and MapReduce). My proposal was to run all
> the tests for any Common change. An HDFS change would only run HDFS
> tests, and any MapReduce change would only run MapReduce tests.
>
> Another thing I didn't mention was that currently Jenkins doesn't run
> tests or apply patches for any changes in hadoop-tools, which would be
> fixed by the change I'm suggesting.
>
> Tom
>
> On Tue, Apr 17, 2012 at 3:17 PM, Giridharan Kesavan
> <[EMAIL PROTECTED]> wrote:
>> I agree with Aaron. Its going increase the test patch build timings
>> significantly which may not be very helpful
>>
>> Im  -1 on this.
>>
>> -Giri
>>
>>
>>
>> On Mon, Apr 16, 2012 at 2:22 PM, Aaron T. Myers <[EMAIL PROTECTED]> wrote:
>>> On Mon, Apr 16, 2012 at 2:14 PM, Alejandro Abdelnur <[EMAIL PROTECTED]>wrote:
>>>
>>>> * all testcases should always be run (else a change in hdfs could
>>>> affect yarn/tools but not be detected, or one in yarn affect tools)
>>>>
>>>
>>> I'm -0 on this suggestion. Yes, it's a nice benefit to check all of the
>>> dependent Hadoop sub-projects for every patch, but it will also
>>> dramatically increase the time test-patch takes to run for any given patch.
>>> In my experience, the vast majority of patches stand little chance of
>>> breaking the dependent sub-projects, making this largely unnecessary and
>>> thus a waste of time and Jenkins build slave resources.
>>>
>>> --
>>> Aaron T. Myers
>>> Software Engineer, Cloudera

--
Alejandro
+
Giridharan Kesavan 2012-04-18, 03:36
+
Alejandro Abdelnur 2012-04-18, 17:11
+
Tom White 2012-04-25, 19:49
+
Aaron T. Myers 2012-04-18, 16:18
+
Radim Kolar 2012-04-27, 09:21
+
Tom White 2012-04-27, 23:48
+
Alejandro Abdelnur 2012-04-27, 23:52
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB