Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # dev >> [DISCUSS] Test on Patch Available


Copy link to this message
-
Re: [DISCUSS] Test on Patch Available
Alright, I have the patch up here:
https://issues.apache.org/jira/browse/FLUME-1787 Once it's committed I
can do the final testing and modifications to our pre-commit build
(https://builds.apache.org/job/PreCommit-FLUME-Build/) and we can turn
this sucker on.

Also, once we have the kinks worked out devs could use this script, I
have been using it to test patches this week.

Brock

On Thu, Dec 20, 2012 at 9:04 AM, Jarek Jarcec Cecho <[EMAIL PROTECTED]> wrote:
> Awesome, thank you very much Brock!
>
> Jarcec
>
> On Dec 19, 2012, at 10:38 PM, Brock Noland <[EMAIL PROTECTED]> wrote:
>
>> Just an update here, the last two comments on this JIRA
>> https://issues.apache.org/jira/browse/FLUME-1787 are the messages on
>> failure and success. At present the only check it does is apply the
>> patch and run the tests. We can add more checks over time.  I ended up
>> not using the Hadoop script because it is one ugly beast of a shell
>> script. The patches on that jira are only for testing this script.
>> I'll be submitting a real patch soon.
>>
>> Brock
>>
>> On Sat, Dec 8, 2012 at 2:39 AM, Mike Percy <[EMAIL PROTECTED]> wrote:
>>> I really like the idea of doing this! +1 from me if we can swing it.
>>>
>>> Regards,
>>> Mike
>>>
>>>
>>> On Fri, Dec 7, 2012 at 3:20 PM, Jarek Jarcec Cecho <[EMAIL PROTECTED]>wrote:
>>>
>>>> Btw I just found very interesting draft that is explaining quite huge
>>>> portions of the design:
>>>>
>>>> http://wiki.apache.org/general/PreCommitBuilds
>>>>
>>>> Jarcec
>>>>
>>>> On Fri, Dec 07, 2012 at 11:14:13AM -0600, Brock Noland wrote:
>>>>> Hi,
>>>>>
>>>>> Great to hear you were thinking the same thing! I have found the
>>>>> script they use to do the checking:
>>>>>
>>>>>
>>>> https://github.com/apache/hadoop-common/blob/trunk/dev-support/test-patch.sh
>>>>>
>>>>> and I think we could probably implement something similar. I agree
>>>>> that our build machines are much busier so I don't think we can use it
>>>>> as a hard requirement for commit. The biggest open question in my mind
>>>>> is how do they query JIRA for patches? Grab the correct patch off of
>>>>> the JIRA? Schedule the job to poll for these things?
>>>>>
>>>>> Brock
>>>>>
>>>>> On Fri, Dec 7, 2012 at 10:26 AM, Jarek Jarcec Cecho <[EMAIL PROTECTED]>
>>>> wrote:
>>>>>> Hi Brock,
>>>>>> that is an excellent idea and I'm definitely in favour of that. In
>>>> fact I wanted to propose something similar myself.
>>>>>>
>>>>>> However I did not quite finished my own investigation yet. It seems
>>>> that this pre commit hooks are done on projects that were mostly donated by
>>>> Yahoo! (Hadoop, Oozie) on Jenkins nodes named "hadoopX". The hadoop named
>>>> nodes have comment "Build machine for Hadoop project builds", so I'm not
>>>> sure that they are available for other projects as well.
>>>>>>
>>>>>> With removing those 9 hadoop notes, it's like 10 other nodes in total
>>>> for entire Apache organization and I do know that sometimes the build queue
>>>> might be quite huge. Having said that it might be good idea to ask Infra
>>>> team if pre-commit hooks are available to all projects or only to projects
>>>> that have "special hardware" available.
>>>>>>
>>>>>> Jarcec
>>>>>>
>>>>>> On Fri, Dec 07, 2012 at 10:11:53AM -0600, Brock Noland wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> I like how in the Hadoop project if you mark a patch "Patch Available"
>>>>>>> jenkins applies the patch to trunk, runs the tests, and outputs the
>>>>>>> result to the JIRA. Here is one example:
>>>>>>>
>>>>>>> http://s.apache.org/qz
>>>>>>>
>>>>>>> I think we should still run tests manually before commit. However,
>>>>>>> this will often give a reviewer an indicator of whether the change
>>>>>>> breaks a test without even having to download the code.
>>>>>>>
>>>>>>> Thoughts?
>>>>>>>
>>>>>>> Brock
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Apache MRUnit - Unit testing MapReduce -
>>>> http://incubator.apache.org/mrunit/
>>>>
>>
>>
>>
>> --
>> Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/

Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/