Hi folks,

Have been toying with the idea of automating the process of running
the tests that constitute the release test plan (the initial tests
would be from https://docs.google.com/spreadsheet/ccc?key=0AvLqcVIqNtlTdE9iemw2STVmWHFVMVVNY2JvZ0NwZGc#gid=0,
and they would be run with/without security turned on in the cluster).
As we continue to develop more system/largescale tests, we would keep
adding them to the harness.

I have set the ball rolling on this within the company and I hope the
community will be interested in seeing such a thing happen.

The idea is to be able to easily run a release artifact through a
series of tests (on our internal cluster or on AWS), and the result
would be published in a machine on AWS (visible to all). In the
future, this could be extended to run the tests on trunk artifacts as
well (catch regressions early).

Thoughts?

Thanks,
Devaraj.

+1

And if you're looking to do this on AWS, I can look into assisting with
funding.
On Wed, Jan 30, 2013 at 10:07 PM, Devaraj Das <[EMAIL PROTECTED]> wrote:

> Hi folks,
>
> Have been toying with the idea of automating the process of running
> the tests that constitute the release test plan (the initial tests
> would be from
> https://docs.google.com/spreadsheet/ccc?key=0AvLqcVIqNtlTdE9iemw2STVmWHFVMVVNY2JvZ0NwZGc#gid=0
> ,
> and they would be run with/without security turned on in the cluster).
> As we continue to develop more system/largescale tests, we would keep
> adding them to the harness.
>
> I have set the ball rolling on this within the company and I hope the
> community will be interested in seeing such a thing happen.
>
> The idea is to be able to easily run a release artifact through a
> series of tests (on our internal cluster or on AWS), and the result
> would be published in a machine on AWS (visible to all). In the
> future, this could be extended to run the tests on trunk artifacts as
> well (catch regressions early).
>
> Thoughts?
>
> Thanks,
> Devaraj.
>

--
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

+1

Sounds very nice.  You going to script the running of the wiki content DD?

You see any overlap with the effort at getting hbase-it tests run on a
continual basis, the stuff discussed here [1] and here [2] (currently a
little stalled till we figure the failed hbase-it test), or as  you see it,
the two efforts compliment each other (which seems to be the case)?

Good stuff,
St.Ack

1.
http://search-hadoop.com/m/6bg8O1ZB7nr&subj=Re+Move+TestReplication+out+of+LargeTests+into+hbase+it+instead+(see
about 2/3rds down the thread)
2.
http://mail-archives.apache.org/mod_mbox/hbase-dev/201212.mbox/%[EMAIL PROTECTED]%3E
On Wed, Jan 30, 2013 at 10:07 PM, Devaraj Das <[EMAIL PROTECTED]> wrote:

> Hi folks,
>
> Have been toying with the idea of automating the process of running
> the tests that constitute the release test plan (the initial tests
> would be from
> https://docs.google.com/spreadsheet/ccc?key=0AvLqcVIqNtlTdE9iemw2STVmWHFVMVVNY2JvZ0NwZGc#gid=0
> ,
> and they would be run with/without security turned on in the cluster).
> As we continue to develop more system/largescale tests, we would keep
> adding them to the harness.
>
> I have set the ball rolling on this within the company and I hope the
> community will be interested in seeing such a thing happen.
>
> The idea is to be able to easily run a release artifact through a
> series of tests (on our internal cluster or on AWS), and the result
> would be published in a machine on AWS (visible to all). In the
> future, this could be extended to run the tests on trunk artifacts as
> well (catch regressions early).
>
> Thoughts?
>
> Thanks,
> Devaraj.
>

Hey Andrew,
Yes I am looking at doing parts of this on AWS and parts in our
internal cluster (since that comes for free) - the AWS component is
there to mostly host the build results, and the tests would run in the
internal cluster (that's my current thinking).
Thanks
Devaraj

On Thu, Jan 31, 2013 at 10:33 AM, Andrew Purtell <[EMAIL PROTECTED]> wrote:
> +1
>
> And if you're looking to do this on AWS, I can look into assisting with
> funding.
>
>
> On Wed, Jan 30, 2013 at 10:07 PM, Devaraj Das <[EMAIL PROTECTED]> wrote:
>
>> Hi folks,
>>
>> Have been toying with the idea of automating the process of running
>> the tests that constitute the release test plan (the initial tests
>> would be from
>> https://docs.google.com/spreadsheet/ccc?key=0AvLqcVIqNtlTdE9iemw2STVmWHFVMVVNY2JvZ0NwZGc#gid=0
>> ,
>> and they would be run with/without security turned on in the cluster).
>> As we continue to develop more system/largescale tests, we would keep
>> adding them to the harness.
>>
>> I have set the ball rolling on this within the company and I hope the
>> community will be interested in seeing such a thing happen.
>>
>> The idea is to be able to easily run a release artifact through a
>> series of tests (on our internal cluster or on AWS), and the result
>> would be published in a machine on AWS (visible to all). In the
>> future, this could be extended to run the tests on trunk artifacts as
>> well (catch regressions early).
>>
>> Thoughts?
>>
>> Thanks,
>> Devaraj.
>>
>
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)

Hi Stack,
We have an internal framework for running the tests (and we might
eventually contribute that), but for now we will run the tests using
the framework, and yes we'd need to convert the release tests (as in
the wiki) to run in that framework.
The framework is kind of like the integration tests framework but I am
thinking of extending it so that it can run the set of
system/integration tests on an hbase artifact present in a certain URL
(let's say some RC) at the click of a button.
In some sense, it is complimentary to the other thread. From what I
remember of Bigtop and the thread, Bigtop supports/builds/tests a
certain released version of HBase, but here the scope is mostly for
catching regressions in yet-to-be-released artifacts / trunk without
actually locking down on any released version.
Thanks
Devaraj

On Thu, Jan 31, 2013 at 12:18 PM, Stack <[EMAIL PROTECTED]> wrote:
> +1
>
> Sounds very nice.  You going to script the running of the wiki content DD?
>
> You see any overlap with the effort at getting hbase-it tests run on a
> continual basis, the stuff discussed here [1] and here [2] (currently a
> little stalled till we figure the failed hbase-it test), or as  you see it,
> the two efforts compliment each other (which seems to be the case)?
>
> Good stuff,
> St.Ack
>
> 1.
> http://search-hadoop.com/m/6bg8O1ZB7nr&subj=Re+Move+TestReplication+out+of+LargeTests+into+hbase+it+instead+(see
> about 2/3rds down the thread)
> 2.
> http://mail-archives.apache.org/mod_mbox/hbase-dev/201212.mbox/%[EMAIL PROTECTED]%3E
>
>
> On Wed, Jan 30, 2013 at 10:07 PM, Devaraj Das <[EMAIL PROTECTED]> wrote:
>
>> Hi folks,
>>
>> Have been toying with the idea of automating the process of running
>> the tests that constitute the release test plan (the initial tests
>> would be from
>> https://docs.google.com/spreadsheet/ccc?key=0AvLqcVIqNtlTdE9iemw2STVmWHFVMVVNY2JvZ0NwZGc#gid=0
>> ,
>> and they would be run with/without security turned on in the cluster).
>> As we continue to develop more system/largescale tests, we would keep
>> adding them to the harness.
>>
>> I have set the ball rolling on this within the company and I hope the
>> community will be interested in seeing such a thing happen.
>>
>> The idea is to be able to easily run a release artifact through a
>> series of tests (on our internal cluster or on AWS), and the result
>> would be published in a machine on AWS (visible to all). In the
>> future, this could be extended to run the tests on trunk artifacts as
>> well (catch regressions early).
>>
>> Thoughts?
>>
>> Thanks,
>> Devaraj.
>>

Shame the tests couldn't also run out in AWS, on suitable instance types
like m2.4xlarge maybe. Then there could be multiple instances of multiple
tests going on. How much coverge how fast would be a function of pooled
resources not just your own. Something to consider?
On Thu, Jan 31, 2013 at 5:21 PM, Devaraj Das <[EMAIL PROTECTED]> wrote:

> Hey Andrew,
> Yes I am looking at doing parts of this on AWS and parts in our
> internal cluster (since that comes for free) - the AWS component is
> there to mostly host the build results, and the tests would run in the
> internal cluster (that's my current thinking).
> Thanks
> Devaraj
>
> On Thu, Jan 31, 2013 at 10:33 AM, Andrew Purtell <[EMAIL PROTECTED]>
> wrote:
> > +1
> >
> > And if you're looking to do this on AWS, I can look into assisting with
> > funding.
> >
> >
> > On Wed, Jan 30, 2013 at 10:07 PM, Devaraj Das <[EMAIL PROTECTED]>
> wrote:
> >
> >> Hi folks,
> >>
> >> Have been toying with the idea of automating the process of running
> >> the tests that constitute the release test plan (the initial tests
> >> would be from
> >>
> https://docs.google.com/spreadsheet/ccc?key=0AvLqcVIqNtlTdE9iemw2STVmWHFVMVVNY2JvZ0NwZGc#gid=0
> >> ,
> >> and they would be run with/without security turned on in the cluster).
> >> As we continue to develop more system/largescale tests, we would keep
> >> adding them to the harness.
> >>
> >> I have set the ball rolling on this within the company and I hope the
> >> community will be interested in seeing such a thing happen.
> >>
> >> The idea is to be able to easily run a release artifact through a
> >> series of tests (on our internal cluster or on AWS), and the result
> >> would be published in a machine on AWS (visible to all). In the
> >> future, this could be extended to run the tests on trunk artifacts as
> >> well (catch regressions early).
> >>
> >> Thoughts?
> >>
> >> Thanks,
> >> Devaraj.
> >>
> >
> >
> >
> > --
> > Best regards,
> >
> >    - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
>

--
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Andrew, agree in principle and technically not a problem. Let me work
with some folks here and get back with some prototype.

On Thu, Jan 31, 2013 at 5:30 PM, Andrew Purtell <[EMAIL PROTECTED]> wrote:
> Shame the tests couldn't also run out in AWS, on suitable instance types
> like m2.4xlarge maybe. Then there could be multiple instances of multiple
> tests going on. How much coverge how fast would be a function of pooled
> resources not just your own. Something to consider?
>
>
> On Thu, Jan 31, 2013 at 5:21 PM, Devaraj Das <[EMAIL PROTECTED]> wrote:
>
>> Hey Andrew,
>> Yes I am looking at doing parts of this on AWS and parts in our
>> internal cluster (since that comes for free) - the AWS component is
>> there to mostly host the build results, and the tests would run in the
>> internal cluster (that's my current thinking).
>> Thanks
>> Devaraj
>>
>> On Thu, Jan 31, 2013 at 10:33 AM, Andrew Purtell <[EMAIL PROTECTED]>
>> wrote:
>> > +1
>> >
>> > And if you're looking to do this on AWS, I can look into assisting with
>> > funding.
>> >
>> >
>> > On Wed, Jan 30, 2013 at 10:07 PM, Devaraj Das <[EMAIL PROTECTED]>
>> wrote:
>> >
>> >> Hi folks,
>> >>
>> >> Have been toying with the idea of automating the process of running
>> >> the tests that constitute the release test plan (the initial tests
>> >> would be from
>> >>
>> https://docs.google.com/spreadsheet/ccc?key=0AvLqcVIqNtlTdE9iemw2STVmWHFVMVVNY2JvZ0NwZGc#gid=0
>> >> ,
>> >> and they would be run with/without security turned on in the cluster).
>> >> As we continue to develop more system/largescale tests, we would keep
>> >> adding them to the harness.
>> >>
>> >> I have set the ball rolling on this within the company and I hope the
>> >> community will be interested in seeing such a thing happen.
>> >>
>> >> The idea is to be able to easily run a release artifact through a
>> >> series of tests (on our internal cluster or on AWS), and the result
>> >> would be published in a machine on AWS (visible to all). In the
>> >> future, this could be extended to run the tests on trunk artifacts as
>> >> well (catch regressions early).
>> >>
>> >> Thoughts?
>> >>
>> >> Thanks,
>> >> Devaraj.
>> >>
>> >
>> >
>> >
>> > --
>> > Best regards,
>> >
>> >    - Andy
>> >
>> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
>> > (via Tom White)
>>
>
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)