Would the HBase dev community be open to alternate hosted Jenkins test
infrastructure?

--
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

What, like travis-ci.org ?

On Tue, Mar 19, 2013 at 8:49 AM, Andrew Purtell <[EMAIL PROTECTED]> wrote:

> Would the HBase dev community be open to alternate hosted Jenkins test
> infrastructure?
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>

I have checked with the folks here and until/unless the situation with ASF
Jenkins VMs improves, we can provide a hosted Jenkins up in EC2, in
us-west-1:

  - A m1.large instance running 24/7 hosting Jenkins

  - A pool of 5 m1.large instances serving as Jenkins slaves, managed by
Jenkins with its EC2 plugin:
https://wiki.jenkins-ci.org/display/JENKINS/Amazon+EC2+Plugin . These would
be launched on demand depending on how many concurrent executors are needed
(we'd set up a 1 executor to 1 instance mapping) and reaped when inactive.
Jenkins' EC2 plugin handles those details.

It would remain to be seen if this is better than the ASF Jenkins VMs, but
if the community is willing to try it, we'd be willing to set it up and
give out access. With EC2's IAM (http://aws.amazon.com/iam/) it's possible
to give all committers accounts for managing these resources. An m1.large
has:
    - 7.5 GiB memory
    - 4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units each)
    - 850 GB instance storage
    - 64-bit platform
    - I/O Performance: Moderate
On Tue, Mar 19, 2013 at 5:23 PM, Nick Dimiduk <[EMAIL PROTECTED]> wrote:

> What, like travis-ci.org ?
>
> On Tue, Mar 19, 2013 at 8:49 AM, Andrew Purtell <[EMAIL PROTECTED]>
> wrote:
>
> > Would the HBase dev community be open to alternate hosted Jenkins test
> > infrastructure?
> >
> > --
> > Best regards,
> >
> >    - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
> >
>

--
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

+1 if can be done.  Can point our CI link over there.  What help you need
Andrew?  Should we try and organize a whiparound among orgs to help pay for
it once its up and running?

I've been on and off working w/ Roman to get our IT builds running over on
bigtop but currently stuck on this strange issue in maven where it compiles
using source and target of 1.4 on his laptop and on bigtop even though it
is taking 1.6 as source and target properties and even though if he cuts
and pastes the command mvn claims to be running into his terminal, it just
works......

St.Ack
On Tue, Mar 19, 2013 at 10:48 AM, Andrew Purtell <[EMAIL PROTECTED]>wrote:

> I have checked with the folks here and until/unless the situation with ASF
> Jenkins VMs improves, we can provide a hosted Jenkins up in EC2, in
> us-west-1:
>
>   - A m1.large instance running 24/7 hosting Jenkins
>
>   - A pool of 5 m1.large instances serving as Jenkins slaves, managed by
> Jenkins with its EC2 plugin:
> https://wiki.jenkins-ci.org/display/JENKINS/Amazon+EC2+Plugin . These
> would
> be launched on demand depending on how many concurrent executors are needed
> (we'd set up a 1 executor to 1 instance mapping) and reaped when inactive.
> Jenkins' EC2 plugin handles those details.
>
> It would remain to be seen if this is better than the ASF Jenkins VMs, but
> if the community is willing to try it, we'd be willing to set it up and
> give out access. With EC2's IAM (http://aws.amazon.com/iam/) it's possible
> to give all committers accounts for managing these resources. An m1.large
> has:
>     - 7.5 GiB memory
>     - 4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units each)
>     - 850 GB instance storage
>     - 64-bit platform
>     - I/O Performance: Moderate
>
>
> On Tue, Mar 19, 2013 at 5:23 PM, Nick Dimiduk <[EMAIL PROTECTED]> wrote:
>
> > What, like travis-ci.org ?
> >
> > On Tue, Mar 19, 2013 at 8:49 AM, Andrew Purtell <[EMAIL PROTECTED]>
> > wrote:
> >
> > > Would the HBase dev community be open to alternate hosted Jenkins test
> > > infrastructure?
> > >
> > > --
> > > Best regards,
> > >
> > >    - Andy
> > >
> > > Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> > > (via Tom White)
> > >
> >
>
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>

+1 Jenkins EC2 plugin works pretty well, though whatever makes a build
machine get "stuck" is equally applicable on EC2.

On Tue, Mar 19, 2013 at 10:48 AM, Andrew Purtell <[EMAIL PROTECTED]>wrote:

> I have checked with the folks here and until/unless the situation with ASF
> Jenkins VMs improves, we can provide a hosted Jenkins up in EC2, in
> us-west-1:
>
>   - A m1.large instance running 24/7 hosting Jenkins
>
>   - A pool of 5 m1.large instances serving as Jenkins slaves, managed by
> Jenkins with its EC2 plugin:
> https://wiki.jenkins-ci.org/display/JENKINS/Amazon+EC2+Plugin . These
> would
> be launched on demand depending on how many concurrent executors are needed
> (we'd set up a 1 executor to 1 instance mapping) and reaped when inactive.
> Jenkins' EC2 plugin handles those details.
>
> It would remain to be seen if this is better than the ASF Jenkins VMs, but
> if the community is willing to try it, we'd be willing to set it up and
> give out access. With EC2's IAM (http://aws.amazon.com/iam/) it's possible
> to give all committers accounts for managing these resources. An m1.large
> has:
>     - 7.5 GiB memory
>     - 4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units each)
>     - 850 GB instance storage
>     - 64-bit platform
>     - I/O Performance: Moderate
>
>
> On Tue, Mar 19, 2013 at 5:23 PM, Nick Dimiduk <[EMAIL PROTECTED]> wrote:
>
> > What, like travis-ci.org ?
> >
> > On Tue, Mar 19, 2013 at 8:49 AM, Andrew Purtell <[EMAIL PROTECTED]>
> > wrote:
> >
> > > Would the HBase dev community be open to alternate hosted Jenkins test
> > > infrastructure?
> > >
> > > --
> > > Best regards,
> > >
> > >    - Andy
> > >
> > > Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> > > (via Tom White)
> > >
> >
>
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>

I'm setting it up now. The Jenkins host is ready, working on the PV AMI for
the slaves.

> Should we try and organize a whiparound among orgs to help pay for it
once its up and running?

Sure, if we pool resources then we could pay for more parallel instances.

We could also consider more interesting (and expensive.. $4/hr/instance)
types for specialized tests: like hi1.4xlarge, which has 60 GB of RAM and 2
TB of SSD backed storage, although they may not be available in us-west-1
yet.
On Tue, Mar 19, 2013 at 8:00 PM, Stack <[EMAIL PROTECTED]> wrote:

> +1 if can be done.  Can point our CI link over there.  What help you need
> Andrew?  Should we try and organize a whiparound among orgs to help pay for
> it once its up and running?
>
> I've been on and off working w/ Roman to get our IT builds running over on
> bigtop but currently stuck on this strange issue in maven where it compiles
> using source and target of 1.4 on his laptop and on bigtop even though it
> is taking 1.6 as source and target properties and even though if he cuts
> and pastes the command mvn claims to be running into his terminal, it just
> works......
>
> St.Ack
>
>
> On Tue, Mar 19, 2013 at 10:48 AM, Andrew Purtell <[EMAIL PROTECTED]
> >wrote:
>
> > I have checked with the folks here and until/unless the situation with
> ASF
> > Jenkins VMs improves, we can provide a hosted Jenkins up in EC2, in
> > us-west-1:
> >
> >   - A m1.large instance running 24/7 hosting Jenkins
> >
> >   - A pool of 5 m1.large instances serving as Jenkins slaves, managed by
> > Jenkins with its EC2 plugin:
> > https://wiki.jenkins-ci.org/display/JENKINS/Amazon+EC2+Plugin . These
> > would
> > be launched on demand depending on how many concurrent executors are
> needed
> > (we'd set up a 1 executor to 1 instance mapping) and reaped when
> inactive.
> > Jenkins' EC2 plugin handles those details.
> >
> > It would remain to be seen if this is better than the ASF Jenkins VMs,
> but
> > if the community is willing to try it, we'd be willing to set it up and
> > give out access. With EC2's IAM (http://aws.amazon.com/iam/) it's
> possible
> > to give all committers accounts for managing these resources. An m1.large
> > has:
> >     - 7.5 GiB memory
> >     - 4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units each)
> >     - 850 GB instance storage
> >     - 64-bit platform
> >     - I/O Performance: Moderate
> >
> >
> > On Tue, Mar 19, 2013 at 5:23 PM, Nick Dimiduk <[EMAIL PROTECTED]>
> wrote:
> >
> > > What, like travis-ci.org ?
> > >
> > > On Tue, Mar 19, 2013 at 8:49 AM, Andrew Purtell <[EMAIL PROTECTED]>
> > > wrote:
> > >
> > > > Would the HBase dev community be open to alternate hosted Jenkins
> test
> > > > infrastructure?
> > > >
> > > > --
> > > > Best regards,
> > > >
> > > >    - Andy
> > > >
> > > > Problems worthy of attack prove their worth by hitting back. - Piet
> > Hein
> > > > (via Tom White)
> > > >
> > >
> >
> >
> >
> > --
> > Best regards,
> >
> >    - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
> >
>

--
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

The current 0.94 run on ubuntu1 has been running for > 80 minutes and has not run a single test, yet (still compiling). This is pretty terrible.
Anything you can do will help :)

________________________________
 From: Andrew Purtell <[EMAIL PROTECTED]>
To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
Sent: Tuesday, March 19, 2013 4:30 PM
Subject: Re: The Jenkins VMs are increasingly slow / overloaded
 
I'm setting it up now. The Jenkins host is ready, working on the PV AMI for
the slaves.

> Should we try and organize a whiparound among orgs to help pay for it
once its up and running?

Sure, if we pool resources then we could pay for more parallel instances.

We could also consider more interesting (and expensive.. $4/hr/instance)
types for specialized tests: like hi1.4xlarge, which has 60 GB of RAM and 2
TB of SSD backed storage, although they may not be available in us-west-1
yet.
On Tue, Mar 19, 2013 at 8:00 PM, Stack <[EMAIL PROTECTED]> wrote:

> +1 if can be done.  Can point our CI link over there.  What help you need
> Andrew?  Should we try and organize a whiparound among orgs to help pay for
> it once its up and running?
>
> I've been on and off working w/ Roman to get our IT builds running over on
> bigtop but currently stuck on this strange issue in maven where it compiles
> using source and target of 1.4 on his laptop and on bigtop even though it
> is taking 1.6 as source and target properties and even though if he cuts
> and pastes the command mvn claims to be running into his terminal, it just
> works......
>
> St.Ack
>
>
> On Tue, Mar 19, 2013 at 10:48 AM, Andrew Purtell <[EMAIL PROTECTED]
> >wrote:
>
> > I have checked with the folks here and until/unless the situation with
> ASF
> > Jenkins VMs improves, we can provide a hosted Jenkins up in EC2, in
> > us-west-1:
> >
> >   - A m1.large instance running 24/7 hosting Jenkins
> >
> >   - A pool of 5 m1.large instances serving as Jenkins slaves, managed by
> > Jenkins with its EC2 plugin:
> > https://wiki.jenkins-ci.org/display/JENKINS/Amazon+EC2+Plugin . These
> > would
> > be launched on demand depending on how many concurrent executors are
> needed
> > (we'd set up a 1 executor to 1 instance mapping) and reaped when
> inactive.
> > Jenkins' EC2 plugin handles those details.
> >
> > It would remain to be seen if this is better than the ASF Jenkins VMs,
> but
> > if the community is willing to try it, we'd be willing to set it up and
> > give out access. With EC2's IAM (http://aws.amazon.com/iam/) it's
> possible
> > to give all committers accounts for managing these resources. An m1.large
> > has:
> >     - 7.5 GiB memory
> >     - 4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units each)
> >     - 850 GB instance storage
> >     - 64-bit platform
> >     - I/O Performance: Moderate
> >
> >
> > On Tue, Mar 19, 2013 at 5:23 PM, Nick Dimiduk <[EMAIL PROTECTED]>
> wrote:
> >
> > > What, like travis-ci.org ?
> > >
> > > On Tue, Mar 19, 2013 at 8:49 AM, Andrew Purtell <[EMAIL PROTECTED]>
> > > wrote:
> > >
> > > > Would the HBase dev community be open to alternate hosted Jenkins
> test
> > > > infrastructure?
> > > >
> > > > --
> > > > Best regards,
> > > >
> > > >    - Andy
> > > >
> > > > Problems worthy of attack prove their worth by hitting back. - Piet
> > Hein
> > > > (via Tom White)
> > > >
> > >
> >
> >
> >
> > --
> > Best regards,
> >
> >    - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
> >
>

--
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

I have this almost ready for you to play around with. Aim is to get a clean
trunk and 0.94 build first before turning it loose. The build deps are all
set up in the custom AMI, ASF build tools, protoc, etc. At this point just
working on debugging the Jenkins SVN plug in. "svn: E155005: No write-lock"
WTF? I may have to switch to using Git.

On Wed, Mar 20, 2013 at 7:33 AM, lars hofhansl <[EMAIL PROTECTED]> wrote:

> The current 0.94 run on ubuntu1 has been running for > 80 minutes and has
> not run a single test, yet (still compiling). This is pretty terrible.
> Anything you can do will help :)
>
>
>
> ________________________________
>  From: Andrew Purtell <[EMAIL PROTECTED]>
> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
> Sent: Tuesday, March 19, 2013 4:30 PM
> Subject: Re: The Jenkins VMs are increasingly slow / overloaded
>
> I'm setting it up now. The Jenkins host is ready, working on the PV AMI for
> the slaves.
>
> > Should we try and organize a whiparound among orgs to help pay for it
> once its up and running?
>
> Sure, if we pool resources then we could pay for more parallel instances.
>
> We could also consider more interesting (and expensive.. $4/hr/instance)
> types for specialized tests: like hi1.4xlarge, which has 60 GB of RAM and 2
> TB of SSD backed storage, although they may not be available in us-west-1
> yet.
>
>
> On Tue, Mar 19, 2013 at 8:00 PM, Stack <[EMAIL PROTECTED]> wrote:
>
> > +1 if can be done.  Can point our CI link over there.  What help you need
> > Andrew?  Should we try and organize a whiparound among orgs to help pay
> for
> > it once its up and running?
> >
> > I've been on and off working w/ Roman to get our IT builds running over
> on
> > bigtop but currently stuck on this strange issue in maven where it
> compiles
> > using source and target of 1.4 on his laptop and on bigtop even though it
> > is taking 1.6 as source and target properties and even though if he cuts
> > and pastes the command mvn claims to be running into his terminal, it
> just
> > works......
> >
> > St.Ack
> >
> >
> > On Tue, Mar 19, 2013 at 10:48 AM, Andrew Purtell <[EMAIL PROTECTED]
> > >wrote:
> >
> > > I have checked with the folks here and until/unless the situation with
> > ASF
> > > Jenkins VMs improves, we can provide a hosted Jenkins up in EC2, in
> > > us-west-1:
> > >
> > >   - A m1.large instance running 24/7 hosting Jenkins
> > >
> > >   - A pool of 5 m1.large instances serving as Jenkins slaves, managed
> by
> > > Jenkins with its EC2 plugin:
> > > https://wiki.jenkins-ci.org/display/JENKINS/Amazon+EC2+Plugin . These
> > > would
> > > be launched on demand depending on how many concurrent executors are
> > needed
> > > (we'd set up a 1 executor to 1 instance mapping) and reaped when
> > inactive.
> > > Jenkins' EC2 plugin handles those details.
> > >
> > > It would remain to be seen if this is better than the ASF Jenkins VMs,
> > but
> > > if the community is willing to try it, we'd be willing to set it up and
> > > give out access. With EC2's IAM (http://aws.amazon.com/iam/) it's
> > possible
> > > to give all committers accounts for managing these resources. An
> m1.large
> > > has:
> > >     - 7.5 GiB memory
> > >     - 4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units
> each)
> > >     - 850 GB instance storage
> > >     - 64-bit platform
> > >     - I/O Performance: Moderate
> > >
> > >
> > > On Tue, Mar 19, 2013 at 5:23 PM, Nick Dimiduk <[EMAIL PROTECTED]>
> > wrote:
> > >
> > > > What, like travis-ci.org ?
> > > >
> > > > On Tue, Mar 19, 2013 at 8:49 AM, Andrew Purtell <[EMAIL PROTECTED]
> >
> > > > wrote:
> > > >
> > > > > Would the HBase dev community be open to alternate hosted Jenkins
> > test
> > > > > infrastructure?
> > > > >
> > > > > --
> > > > > Best regards,
> > > > >
> > > > >    - Andy
> > > > >
> > > > > Problems worthy of attack prove their worth by hitting back. - Piet
> > > Hein
> > > > > (via Tom White)
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Best regards,

Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

We have an EC2 hosted Jenkins master and slave pool set up in us-west-1. There
are 'HBase-TRUNK' and 'HBase-0.94' projects with bare bones configuration. I've
mailed access information and credentials to stack, larsh, and ndimiduk --
a list drawn up simply based on replies to this thread. Committers can
contact me or any of them for accounts if you'd like to help gently coax
this into a useful state.

Once it's useful I'd imagine an announcement here of a new DNS A record
under hbase.org somewhere pointing to it.
On Wed, Mar 20, 2013 at 9:57 AM, Andrew Purtell <[EMAIL PROTECTED]> wrote:

> I have this almost ready for you to play around with.
>

--
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Thanks Andrew for this effort.

Can I get access as well ?

On Wed, Mar 20, 2013 at 2:37 PM, Andrew Purtell <[EMAIL PROTECTED]> wrote:

> We have an EC2 hosted Jenkins master and slave pool set up in us-west-1.
> There
> are 'HBase-TRUNK' and 'HBase-0.94' projects with bare bones configuration.
> I've
> mailed access information and credentials to stack, larsh, and ndimiduk --
> a list drawn up simply based on replies to this thread. Committers can
> contact me or any of them for accounts if you'd like to help gently coax
> this into a useful state.
>
> Once it's useful I'd imagine an announcement here of a new DNS A record
> under hbase.org somewhere pointing to it.
>
>
> On Wed, Mar 20, 2013 at 9:57 AM, Andrew Purtell <[EMAIL PROTECTED]>
> wrote:
>
> > I have this almost ready for you to play around with.
> >
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>

Go to http://54.241.6.143  Log in as user "yuzhihong" current password
"PravAZa2".
On Wed, Mar 20, 2013 at 10:39 PM, Ted Yu <[EMAIL PROTECTED]> wrote:

> Thanks Andrew for this effort.
>
> Can I get access as well ?
>
> On Wed, Mar 20, 2013 at 2:37 PM, Andrew Purtell <[EMAIL PROTECTED]>
> wrote:
>
> > We have an EC2 hosted Jenkins master and slave pool set up in us-west-1.
> > There
> > are 'HBase-TRUNK' and 'HBase-0.94' projects with bare bones
> configuration.
> > I've
> > mailed access information and credentials to stack, larsh, and ndimiduk
> --
> > a list drawn up simply based on replies to this thread. Committers can
> > contact me or any of them for accounts if you'd like to help gently coax
> > this into a useful state.
> >
> > Once it's useful I'd imagine an announcement here of a new DNS A record
> > under hbase.org somewhere pointing to it.
> >
> >
> > On Wed, Mar 20, 2013 at 9:57 AM, Andrew Purtell <[EMAIL PROTECTED]>
> > wrote:
> >
> > > I have this almost ready for you to play around with.
> > >
> >
> > --
> > Best regards,
> >
> >    - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
> >
>

--
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Drat, wrong recipient. Well, that won't work now anyway. :-)
On Wed, Mar 20, 2013 at 10:44 PM, Andrew Purtell <[EMAIL PROTECTED]>wrote:

> Go to http://54.241.6.143  Log in as user "yuzhihong" current password
> "PravAZa2".
>
>
> On Wed, Mar 20, 2013 at 10:39 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
>> Thanks Andrew for this effort.
>>
>> Can I get access as well ?
>>
>> On Wed, Mar 20, 2013 at 2:37 PM, Andrew Purtell <[EMAIL PROTECTED]>
>> wrote:
>>
>> > We have an EC2 hosted Jenkins master and slave pool set up in us-west-1.
>> > There
>> > are 'HBase-TRUNK' and 'HBase-0.94' projects with bare bones
>> configuration.
>> > I've
>> > mailed access information and credentials to stack, larsh, and ndimiduk
>> --
>> > a list drawn up simply based on replies to this thread. Committers can
>> > contact me or any of them for accounts if you'd like to help gently coax
>> > this into a useful state.
>> >
>> > Once it's useful I'd imagine an announcement here of a new DNS A record
>> > under hbase.org somewhere pointing to it.
>> >
>> >
>> > On Wed, Mar 20, 2013 at 9:57 AM, Andrew Purtell <[EMAIL PROTECTED]>
>> > wrote:
>> >
>> > > I have this almost ready for you to play around with.
>> > >
>> >
>> > --
>> > Best regards,
>> >
>> >    - Andy
>> >
>> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
>> > (via Tom White)
>> >
>>
>
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>

--
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

This machine is now running builds: http://54.241.6.143/    After some
messing to fix memory allocation, it seems tests are failing legitimately
now for both 0.94 and trunk.   They warrant fixing (thanks for setting this
up Andrew).
St.Ack
On Tue, Mar 19, 2013 at 10:48 AM, Andrew Purtell <[EMAIL PROTECTED]>wrote:

> I have checked with the folks here and until/unless the situation with ASF
> Jenkins VMs improves, we can provide a hosted Jenkins up in EC2, in
> us-west-1:
>
>   - A m1.large instance running 24/7 hosting Jenkins
>
>   - A pool of 5 m1.large instances serving as Jenkins slaves, managed by
> Jenkins with its EC2 plugin:
> https://wiki.jenkins-ci.org/display/JENKINS/Amazon+EC2+Plugin . These
> would
> be launched on demand depending on how many concurrent executors are needed
> (we'd set up a 1 executor to 1 instance mapping) and reaped when inactive.
> Jenkins' EC2 plugin handles those details.
>
> It would remain to be seen if this is better than the ASF Jenkins VMs, but
> if the community is willing to try it, we'd be willing to set it up and
> give out access. With EC2's IAM (http://aws.amazon.com/iam/) it's possible
> to give all committers accounts for managing these resources. An m1.large
> has:
>     - 7.5 GiB memory
>     - 4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units each)
>     - 850 GB instance storage
>     - 64-bit platform
>     - I/O Performance: Moderate
>
>
> On Tue, Mar 19, 2013 at 5:23 PM, Nick Dimiduk <[EMAIL PROTECTED]> wrote:
>
> > What, like travis-ci.org ?
> >
> > On Tue, Mar 19, 2013 at 8:49 AM, Andrew Purtell <[EMAIL PROTECTED]>
> > wrote:
> >
> > > Would the HBase dev community be open to alternate hosted Jenkins test
> > > infrastructure?
> > >
> > > --
> > > Best regards,
> > >
> > >    - Andy
> > >
> > > Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> > > (via Tom White)
> > >
> >
>
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>

Who added all the new hbase-on-hadoop2 builds and on-security builds?  Was
it you Andrew?
St.Ack
On Thu, Mar 21, 2013 at 9:38 PM, Stack <[EMAIL PROTECTED]> wrote:

> This machine is now running builds: http://54.241.6.143/    After some
> messing to fix memory allocation, it seems tests are failing legitimately
> now for both 0.94 and trunk.   They warrant fixing (thanks for setting this
> up Andrew).
> St.Ack
>
>
> On Tue, Mar 19, 2013 at 10:48 AM, Andrew Purtell <[EMAIL PROTECTED]>wrote:
>
>> I have checked with the folks here and until/unless the situation with ASF
>> Jenkins VMs improves, we can provide a hosted Jenkins up in EC2, in
>> us-west-1:
>>
>>   - A m1.large instance running 24/7 hosting Jenkins
>>
>>   - A pool of 5 m1.large instances serving as Jenkins slaves, managed by
>> Jenkins with its EC2 plugin:
>> https://wiki.jenkins-ci.org/display/JENKINS/Amazon+EC2+Plugin . These
>> would
>> be launched on demand depending on how many concurrent executors are
>> needed
>> (we'd set up a 1 executor to 1 instance mapping) and reaped when inactive.
>> Jenkins' EC2 plugin handles those details.
>>
>> It would remain to be seen if this is better than the ASF Jenkins VMs, but
>> if the community is willing to try it, we'd be willing to set it up and
>> give out access. With EC2's IAM (http://aws.amazon.com/iam/) it's
>> possible
>> to give all committers accounts for managing these resources. An m1.large
>> has:
>>     - 7.5 GiB memory
>>     - 4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units each)
>>     - 850 GB instance storage
>>     - 64-bit platform
>>     - I/O Performance: Moderate
>>
>>
>> On Tue, Mar 19, 2013 at 5:23 PM, Nick Dimiduk <[EMAIL PROTECTED]> wrote:
>>
>> > What, like travis-ci.org ?
>> >
>> > On Tue, Mar 19, 2013 at 8:49 AM, Andrew Purtell <[EMAIL PROTECTED]>
>> > wrote:
>> >
>> > > Would the HBase dev community be open to alternate hosted Jenkins test
>> > > infrastructure?
>> > >
>> > > --
>> > > Best regards,
>> > >
>> > >    - Andy
>> > >
>> > > Problems worthy of attack prove their worth by hitting back. - Piet
>> Hein
>> > > (via Tom White)
>> > >
>> >
>>
>>
>>
>> --
>> Best regards,
>>
>>    - Andy
>>
>> Problems worthy of attack prove their worth by hitting back. - Piet Hein
>> (via Tom White)
>>
>
>

It was me. If they are stomping on something you are doing please feel free
to disable or delete them.

On Saturday, March 23, 2013, Stack wrote:

> Who added all the new hbase-on-hadoop2 builds and on-security builds?  Was
> it you Andrew?
> St.Ack
>
>
> On Thu, Mar 21, 2013 at 9:38 PM, Stack <[EMAIL PROTECTED] <javascript:;>>
> wrote:
>
> > This machine is now running builds: http://54.241.6.143/    After some
> > messing to fix memory allocation, it seems tests are failing legitimately
> > now for both 0.94 and trunk.   They warrant fixing (thanks for setting
> this
> > up Andrew).
> > St.Ack
> >
> >
> > On Tue, Mar 19, 2013 at 10:48 AM, Andrew Purtell <[EMAIL PROTECTED]<javascript:;>
> >wrote:
> >
> >> I have checked with the folks here and until/unless the situation with
> ASF
> >> Jenkins VMs improves, we can provide a hosted Jenkins up in EC2, in
> >> us-west-1:
> >>
> >>   - A m1.large instance running 24/7 hosting Jenkins
> >>
> >>   - A pool of 5 m1.large instances serving as Jenkins slaves, managed by
> >> Jenkins with its EC2 plugin:
> >> https://wiki.jenkins-ci.org/display/JENKINS/Amazon+EC2+Plugin . These
> >> would
> >> be launched on demand depending on how many concurrent executors are
> >> needed
> >> (we'd set up a 1 executor to 1 instance mapping) and reaped when
> inactive.
> >> Jenkins' EC2 plugin handles those details.
> >>
> >> It would remain to be seen if this is better than the ASF Jenkins VMs,
> but
> >> if the community is willing to try it, we'd be willing to set it up and
> >> give out access. With EC2's IAM (http://aws.amazon.com/iam/) it's
> >> possible
> >> to give all committers accounts for managing these resources. An
> m1.large
> >> has:
> >>     - 7.5 GiB memory
> >>     - 4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units
> each)
> >>     - 850 GB instance storage
> >>     - 64-bit platform
> >>     - I/O Performance: Moderate
> >>
> >>
> >> On Tue, Mar 19, 2013 at 5:23 PM, Nick Dimiduk <[EMAIL PROTECTED]<javascript:;>>
> wrote:
> >>
> >> > What, like travis-ci.org ?
> >> >
> >> > On Tue, Mar 19, 2013 at 8:49 AM, Andrew Purtell <[EMAIL PROTECTED]<javascript:;>
> >
> >> > wrote:
> >> >
> >> > > Would the HBase dev community be open to alternate hosted Jenkins
> test
> >> > > infrastructure?
> >> > >
> >> > > --
> >> > > Best regards,
> >> > >
> >> > >    - Andy
> >> > >
> >> > > Problems worthy of attack prove their worth by hitting back. - Piet
> >> Hein
> >> > > (via Tom White)
> >> > >
> >> >
> >>
> >>
> >>
> >> --
> >> Best regards,
> >>
> >>    - Andy
> >>
> >> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> >> (via Tom White)
> >>
> >
> >
>
--
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

On Sat, Mar 23, 2013 at 3:11 AM, Andrew Purtell <[EMAIL PROTECTED]> wrote:

> It was me. If they are stomping on something you are doing please feel free
> to disable or delete them.
>
>
No worries.  Was just wondering.  They are all good builds to have.  I
tried to get it to email dev list but doesn't seem to be working... No
mailer?  Its finding good stuff, broke tests.  Having a 'second opinion'
helps.

St.Ack

We got an alert from AWS about SMTP traffic out of that instance. Seems we have to apply for permission to send emails out directly. For now they are blocking it. I'm working on that.

On Mar 23, 2013, at 9:24 PM, Stack <[EMAIL PROTECTED]> wrote:

> On Sat, Mar 23, 2013 at 3:11 AM, Andrew Purtell <[EMAIL PROTECTED]> wrote:
>
>> It was me. If they are stomping on something you are doing please feel free
>> to disable or delete them.
> No worries.  Was just wondering.  They are all good builds to have.  I
> tried to get it to email dev list but doesn't seem to be working... No
> mailer?  Its finding good stuff, broke tests.  Having a 'second opinion'
> helps.
>
> St.Ack

I found that Maven was being killed on the slaves by the Linux OOM killer
sometimes for >= 0.95. Seems the m1.large process didn't have enough memory
to host the Jenkins slave, Maven with its 3G+ heap, and the forked JVMs for
the medium and large tests at the same time. Switching to the m1.xlarge
type resolved this. Now the 0.95 and trunk builds fail for what looks like
a legitimate problem with a hanging test.
On Sat, Mar 23, 2013 at 9:24 PM, Stack <[EMAIL PROTECTED]> wrote:

> On Sat, Mar 23, 2013 at 3:11 AM, Andrew Purtell <[EMAIL PROTECTED]>
> wrote:
>
> > It was me. If they are stomping on something you are doing please feel
> free
> > to disable or delete them.
> >
> >
> No worries.  Was just wondering.  They are all good builds to have.  I
> tried to get it to email dev list but doesn't seem to be working... No
> mailer?  Its finding good stuff, broke tests.  Having a 'second opinion'
> helps.
>
> St.Ack
>

--
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

The HBase 0.94 build is now testing green!
http://54.241.6.143/job/HBase-0.94/

AWS approved our request to allow outbound SMTP, so it should be possible
to set up this Jenkins to mail dev@ now.
On Tue, Mar 26, 2013 at 1:47 AM, Andrew Purtell <[EMAIL PROTECTED]> wrote:

> I found that Maven was being killed on the slaves by the Linux OOM killer
> sometimes for >= 0.95. Seems the m1.large process didn't have enough memory
> to host the Jenkins slave, Maven with its 3G+ heap, and the forked JVMs for
> the medium and large tests at the same time. Switching to the m1.xlarge
> type resolved this. Now the 0.95 and trunk builds fail for what looks like
> a legitimate problem with a hanging test.
>

--
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Good on you Andrew.

Now to make 0.95 and trunk green.

St.Ack
On Tue, Mar 26, 2013 at 1:28 PM, Andrew Purtell <[EMAIL PROTECTED]> wrote:

> The HBase 0.94 build is now testing green!
> http://54.241.6.143/job/HBase-0.94/
>
> AWS approved our request to allow outbound SMTP, so it should be possible
> to set up this Jenkins to mail dev@ now.
>
>
> On Tue, Mar 26, 2013 at 1:47 AM, Andrew Purtell <[EMAIL PROTECTED]>
> wrote:
>
> > I found that Maven was being killed on the slaves by the Linux OOM killer
> > sometimes for >= 0.95. Seems the m1.large process didn't have enough
> memory
> > to host the Jenkins slave, Maven with its 3G+ heap, and the forked JVMs
> for
> > the medium and large tests at the same time. Switching to the m1.xlarge
> > type resolved this. Now the 0.95 and trunk builds fail for what looks
> like
> > a legitimate problem with a hanging test.
> >
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>

Ah well short lived, it's red again, but is probably a real issue. Will dig
in.
On Tue, Mar 26, 2013 at 10:59 PM, Stack <[EMAIL PROTECTED]> wrote:

> Good on you Andrew.
>
> Now to make 0.95 and trunk green.
>
> St.Ack
>
>
> On Tue, Mar 26, 2013 at 1:28 PM, Andrew Purtell <[EMAIL PROTECTED]>
> wrote:
>
> > The HBase 0.94 build is now testing green!
> > http://54.241.6.143/job/HBase-0.94/
> >
> > AWS approved our request to allow outbound SMTP, so it should be possible
> > to set up this Jenkins to mail dev@ now.
> >
> >
> > On Tue, Mar 26, 2013 at 1:47 AM, Andrew Purtell <[EMAIL PROTECTED]>
> > wrote:
> >
> > > I found that Maven was being killed on the slaves by the Linux OOM
> killer
> > > sometimes for >= 0.95. Seems the m1.large process didn't have enough
> > memory
> > > to host the Jenkins slave, Maven with its 3G+ heap, and the forked JVMs
> > for
> > > the medium and large tests at the same time. Switching to the m1.xlarge
> > > type resolved this. Now the 0.95 and trunk builds fail for what looks
> > like
> > > a legitimate problem with a hanging test.
> > >
> >
> > --
> > Best regards,
> >
> >    - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
> >
>

--
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

On Tue, Mar 26, 2013 at 1:28 PM, Andrew Purtell <[EMAIL PROTECTED]> wrote:

> The HBase 0.94 build is now testing green!
> http://54.241.6.143/job/HBase-0.94/
>

^5!

On Tue, Mar 26, 2013 at 1:47 AM, Andrew Purtell <[EMAIL PROTECTED]> wrote:
>
> > I found that Maven was being killed on the slaves by the Linux OOM killer
> > sometimes for >= 0.95. Seems the m1.large process didn't have enough
> memory
> > to host the Jenkins slave, Maven with its 3G+ heap, and the forked JVMs
> for
> > the medium and large tests at the same time. Switching to the m1.xlarge
> > type resolved this. Now the 0.95 and trunk builds fail for what looks
> like
> > a legitimate problem with a hanging test.
> >
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>

In general moving from using the m1.large (2 vcores, 7.5 GB RAM) to the
m1.xlarge (4 vcores, 15 GB RAM) instance type for the slaves helped with a
build/test timeout, so now I'd about claim the test environment is sane. We
are now seeing that replication tests are flapping, occasionally timing out
internally:

See
http://54.241.6.143/job/HBase-0.94/org.apache.hbase$hbase/24/testReport/junit/org.apache.hadoop.hbase.replication/TestReplicationQueueFailoverCompressed/queueFailover/
and
http://54.241.6.143/job/HBase-0.94-Security/org.apache.hbase$hbase/7/testReport/junit/org.apache.hadoop.hbase.replication/TestReplicationQueueFailover/queueFailover/
The 0.94 and 0.94-security builds are alternating between green and red as
a result.

Perhaps we should reopen/revisit either adjusting the internal timeouts for
these tests or the other JIRA about moving minicluster replication tests to
hbase-it.
On Wed, Mar 27, 2013 at 1:49 AM, Nick Dimiduk <[EMAIL PROTECTED]> wrote:

> On Tue, Mar 26, 2013 at 1:28 PM, Andrew Purtell <[EMAIL PROTECTED]>
> wrote:
>
> > The HBase 0.94 build is now testing green!
> > http://54.241.6.143/job/HBase-0.94/
> >
>
> ^5!
>
> On Tue, Mar 26, 2013 at 1:47 AM, Andrew Purtell <[EMAIL PROTECTED]>
> wrote:
> >
> > > I found that Maven was being killed on the slaves by the Linux OOM
> killer
> > > sometimes for >= 0.95. Seems the m1.large process didn't have enough
> > memory
> > > to host the Jenkins slave, Maven with its 3G+ heap, and the forked JVMs
> > for
> > > the medium and large tests at the same time. Switching to the m1.xlarge
> > > type resolved this. Now the 0.95 and trunk builds fail for what looks
> > like
> > > a legitimate problem with a hanging test.
> > >
> >
> > --
> > Best regards,
> >
> >    - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
> >
>

--
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Trunk and 0.95 builds are not in good shape.
0.95 builds have been failing for 32 times.

On Apache Jenkins, looks like TestAssignmentManagerOnCluster has failed
quite often for 0.95 and trunk builds.

On Wed, Mar 27, 2013 at 7:18 AM, Andrew Purtell <[EMAIL PROTECTED]> wrote:

> In general moving from using the m1.large (2 vcores, 7.5 GB RAM) to the
> m1.xlarge (4 vcores, 15 GB RAM) instance type for the slaves helped with a
> build/test timeout, so now I'd about claim the test environment is sane. We
> are now seeing that replication tests are flapping, occasionally timing out
> internally:
>
> See
>
> http://54.241.6.143/job/HBase-0.94/org.apache.hbase$hbase/24/testReport/junit/org.apache.hadoop.hbase.replication/TestReplicationQueueFailoverCompressed/queueFailover/
>
>
> and
>
> http://54.241.6.143/job/HBase-0.94-Security/org.apache.hbase$hbase/7/testReport/junit/org.apache.hadoop.hbase.replication/TestReplicationQueueFailover/queueFailover/
>
>
> The 0.94 and 0.94-security builds are alternating between green and red as
> a result.
>
> Perhaps we should reopen/revisit either adjusting the internal timeouts for
> these tests or the other JIRA about moving minicluster replication tests to
> hbase-it.
>
>
> On Wed, Mar 27, 2013 at 1:49 AM, Nick Dimiduk <[EMAIL PROTECTED]> wrote:
>
> > On Tue, Mar 26, 2013 at 1:28 PM, Andrew Purtell <[EMAIL PROTECTED]>
> > wrote:
> >
> > > The HBase 0.94 build is now testing green!
> > > http://54.241.6.143/job/HBase-0.94/
> > >
> >
> > ^5!
> >
> > On Tue, Mar 26, 2013 at 1:47 AM, Andrew Purtell <[EMAIL PROTECTED]>
> > wrote:
> > >
> > > > I found that Maven was being killed on the slaves by the Linux OOM
> > killer
> > > > sometimes for >= 0.95. Seems the m1.large process didn't have enough
> > > memory
> > > > to host the Jenkins slave, Maven with its 3G+ heap, and the forked
> JVMs
> > > for
> > > > the medium and large tests at the same time. Switching to the
> m1.xlarge
> > > > type resolved this. Now the 0.95 and trunk builds fail for what looks
> > > like
> > > > a legitimate problem with a hanging test.
> > > >
> > >
> > > --
> > > Best regards,
> > >
> > >    - Andy
> > >
> > > Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> > > (via Tom White)
> > >
> >
>
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>

True, but unlike 0.94 the state of 0.95 and trunk is impacted by Stack's
wrangling with Maven to find a sane site and assembly, a number of build
failures are due to that. Also you'll note that prior to yesterday the
Linux OOM killer was nuking the bloated Maven processes on the build
slaves. Let's give these builds a bit of time for this stuff to get sorted
out. The failures in 0.94 seem immediately actionable.
On Wed, Mar 27, 2013 at 3:38 PM, Ted Yu <[EMAIL PROTECTED]> wrote:

> Trunk and 0.95 builds are not in good shape.
> 0.95 builds have been failing for 32 times.
>
> On Apache Jenkins, looks like TestAssignmentManagerOnCluster has failed
> quite often for 0.95 and trunk builds.
>
> On Wed, Mar 27, 2013 at 7:18 AM, Andrew Purtell <[EMAIL PROTECTED]>
> wrote:
>
> > In general moving from using the m1.large (2 vcores, 7.5 GB RAM) to the
> > m1.xlarge (4 vcores, 15 GB RAM) instance type for the slaves helped with
> a
> > build/test timeout, so now I'd about claim the test environment is sane.
> We
> > are now seeing that replication tests are flapping, occasionally timing
> out
> > internally:
> >
> > See
> >
> >
> http://54.241.6.143/job/HBase-0.94/org.apache.hbase$hbase/24/testReport/junit/org.apache.hadoop.hbase.replication/TestReplicationQueueFailoverCompressed/queueFailover/
> >
> >
> > and
> >
> >
> http://54.241.6.143/job/HBase-0.94-Security/org.apache.hbase$hbase/7/testReport/junit/org.apache.hadoop.hbase.replication/TestReplicationQueueFailover/queueFailover/
> >
> >
> > The 0.94 and 0.94-security builds are alternating between green and red
> as
> > a result.
> >
> > Perhaps we should reopen/revisit either adjusting the internal timeouts
> for
> > these tests or the other JIRA about moving minicluster replication tests
> to
> > hbase-it.
> >
> >
> > On Wed, Mar 27, 2013 at 1:49 AM, Nick Dimiduk <[EMAIL PROTECTED]>
> wrote:
> >
> > > On Tue, Mar 26, 2013 at 1:28 PM, Andrew Purtell <[EMAIL PROTECTED]>
> > > wrote:
> > >
> > > > The HBase 0.94 build is now testing green!
> > > > http://54.241.6.143/job/HBase-0.94/
> > > >
> > >
> > > ^5!
> > >
> > > On Tue, Mar 26, 2013 at 1:47 AM, Andrew Purtell <[EMAIL PROTECTED]>
> > > wrote:
> > > >
> > > > > I found that Maven was being killed on the slaves by the Linux OOM
> > > killer
> > > > > sometimes for >= 0.95. Seems the m1.large process didn't have
> enough
> > > > memory
> > > > > to host the Jenkins slave, Maven with its 3G+ heap, and the forked
> > JVMs
> > > > for
> > > > > the medium and large tests at the same time. Switching to the
> > m1.xlarge
> > > > > type resolved this. Now the 0.95 and trunk builds fail for what
> looks
> > > > like
> > > > > a legitimate problem with a hanging test.
> > > > >
> > > >
> > > > --
> > > > Best regards,
> > > >
> > > >    - Andy
> > > >
> > > > Problems worthy of attack prove their worth by hitting back. - Piet
> > Hein
> > > > (via Tom White)
> > > >
> > >
> >
> >
> >
> > --
> > Best regards,
> >
> >    - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
> >
>

--
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Looks like Apache Jenkins went off several times this week.

Is it difficult to hook up patching test with the new Jenkins ?

Thanks

On Wed, Mar 27, 2013 at 7:49 AM, Andrew Purtell <[EMAIL PROTECTED]> wrote:

> True, but unlike 0.94 the state of 0.95 and trunk is impacted by Stack's
> wrangling with Maven to find a sane site and assembly, a number of build
> failures are due to that. Also you'll note that prior to yesterday the
> Linux OOM killer was nuking the bloated Maven processes on the build
> slaves. Let's give these builds a bit of time for this stuff to get sorted
> out. The failures in 0.94 seem immediately actionable.
>
>
> On Wed, Mar 27, 2013 at 3:38 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
> > Trunk and 0.95 builds are not in good shape.
> > 0.95 builds have been failing for 32 times.
> >
> > On Apache Jenkins, looks like TestAssignmentManagerOnCluster has failed
> > quite often for 0.95 and trunk builds.
> >
> > On Wed, Mar 27, 2013 at 7:18 AM, Andrew Purtell <[EMAIL PROTECTED]>
> > wrote:
> >
> > > In general moving from using the m1.large (2 vcores, 7.5 GB RAM) to the
> > > m1.xlarge (4 vcores, 15 GB RAM) instance type for the slaves helped
> with
> > a
> > > build/test timeout, so now I'd about claim the test environment is
> sane.
> > We
> > > are now seeing that replication tests are flapping, occasionally timing
> > out
> > > internally:
> > >
> > > See
> > >
> > >
> >
> http://54.241.6.143/job/HBase-0.94/org.apache.hbase$hbase/24/testReport/junit/org.apache.hadoop.hbase.replication/TestReplicationQueueFailoverCompressed/queueFailover/
> > >
> > >
> > > and
> > >
> > >
> >
> http://54.241.6.143/job/HBase-0.94-Security/org.apache.hbase$hbase/7/testReport/junit/org.apache.hadoop.hbase.replication/TestReplicationQueueFailover/queueFailover/
> > >
> > >
> > > The 0.94 and 0.94-security builds are alternating between green and red
> > as
> > > a result.
> > >
> > > Perhaps we should reopen/revisit either adjusting the internal timeouts
> > for
> > > these tests or the other JIRA about moving minicluster replication
> tests
> > to
> > > hbase-it.
> > >
> > >
> > > On Wed, Mar 27, 2013 at 1:49 AM, Nick Dimiduk <[EMAIL PROTECTED]>
> > wrote:
> > >
> > > > On Tue, Mar 26, 2013 at 1:28 PM, Andrew Purtell <[EMAIL PROTECTED]
> >
> > > > wrote:
> > > >
> > > > > The HBase 0.94 build is now testing green!
> > > > > http://54.241.6.143/job/HBase-0.94/
> > > > >
> > > >
> > > > ^5!
> > > >
> > > > On Tue, Mar 26, 2013 at 1:47 AM, Andrew Purtell <[EMAIL PROTECTED]
> >
> > > > wrote:
> > > > >
> > > > > > I found that Maven was being killed on the slaves by the Linux
> OOM
> > > > killer
> > > > > > sometimes for >= 0.95. Seems the m1.large process didn't have
> > enough
> > > > > memory
> > > > > > to host the Jenkins slave, Maven with its 3G+ heap, and the
> forked
> > > JVMs
> > > > > for
> > > > > > the medium and large tests at the same time. Switching to the
> > > m1.xlarge
> > > > > > type resolved this. Now the 0.95 and trunk builds fail for what
> > looks
> > > > > like
> > > > > > a legitimate problem with a hanging test.
> > > > > >
> > > > >
> > > > > --
> > > > > Best regards,
> > > > >
> > > > >    - Andy
> > > > >
> > > > > Problems worthy of attack prove their worth by hitting back. - Piet
> > > Hein
> > > > > (via Tom White)
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Best regards,
> > >
> > >    - Andy
> > >
> > > Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> > > (via Tom White)
> > >
> >
>
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>

Looks like 4 ECs Jenkins slaves are offline at the moment ...
On Wed, Mar 27, 2013 at 1:19 PM, Ted Yu <[EMAIL PROTECTED]> wrote:

> Looks like Apache Jenkins went off several times this week.
>
> Is it difficult to hook up patching test with the new Jenkins ?
>
> Thanks
>
>
> On Wed, Mar 27, 2013 at 7:49 AM, Andrew Purtell <[EMAIL PROTECTED]>wrote:
>
>> True, but unlike 0.94 the state of 0.95 and trunk is impacted by Stack's
>> wrangling with Maven to find a sane site and assembly, a number of build
>> failures are due to that. Also you'll note that prior to yesterday the
>> Linux OOM killer was nuking the bloated Maven processes on the build
>> slaves. Let's give these builds a bit of time for this stuff to get sorted
>> out. The failures in 0.94 seem immediately actionable.
>>
>>
>> On Wed, Mar 27, 2013 at 3:38 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>>
>> > Trunk and 0.95 builds are not in good shape.
>> > 0.95 builds have been failing for 32 times.
>> >
>> > On Apache Jenkins, looks like TestAssignmentManagerOnCluster has failed
>> > quite often for 0.95 and trunk builds.
>> >
>> > On Wed, Mar 27, 2013 at 7:18 AM, Andrew Purtell <[EMAIL PROTECTED]>
>> > wrote:
>> >
>> > > In general moving from using the m1.large (2 vcores, 7.5 GB RAM) to
>> the
>> > > m1.xlarge (4 vcores, 15 GB RAM) instance type for the slaves helped
>> with
>> > a
>> > > build/test timeout, so now I'd about claim the test environment is
>> sane.
>> > We
>> > > are now seeing that replication tests are flapping, occasionally
>> timing
>> > out
>> > > internally:
>> > >
>> > > See
>> > >
>> > >
>> >
>> http://54.241.6.143/job/HBase-0.94/org.apache.hbase$hbase/24/testReport/junit/org.apache.hadoop.hbase.replication/TestReplicationQueueFailoverCompressed/queueFailover/
>> > >
>> > >
>> > > and
>> > >
>> > >
>> >
>> http://54.241.6.143/job/HBase-0.94-Security/org.apache.hbase$hbase/7/testReport/junit/org.apache.hadoop.hbase.replication/TestReplicationQueueFailover/queueFailover/
>> > >
>> > >
>> > > The 0.94 and 0.94-security builds are alternating between green and
>> red
>> > as
>> > > a result.
>> > >
>> > > Perhaps we should reopen/revisit either adjusting the internal
>> timeouts
>> > for
>> > > these tests or the other JIRA about moving minicluster replication
>> tests
>> > to
>> > > hbase-it.
>> > >
>> > >
>> > > On Wed, Mar 27, 2013 at 1:49 AM, Nick Dimiduk <[EMAIL PROTECTED]>
>> > wrote:
>> > >
>> > > > On Tue, Mar 26, 2013 at 1:28 PM, Andrew Purtell <
>> [EMAIL PROTECTED]>
>> > > > wrote:
>> > > >
>> > > > > The HBase 0.94 build is now testing green!
>> > > > > http://54.241.6.143/job/HBase-0.94/
>> > > > >
>> > > >
>> > > > ^5!
>> > > >
>> > > > On Tue, Mar 26, 2013 at 1:47 AM, Andrew Purtell <
>> [EMAIL PROTECTED]>
>> > > > wrote:
>> > > > >
>> > > > > > I found that Maven was being killed on the slaves by the Linux
>> OOM
>> > > > killer
>> > > > > > sometimes for >= 0.95. Seems the m1.large process didn't have
>> > enough
>> > > > > memory
>> > > > > > to host the Jenkins slave, Maven with its 3G+ heap, and the
>> forked
>> > > JVMs
>> > > > > for
>> > > > > > the medium and large tests at the same time. Switching to the
>> > > m1.xlarge
>> > > > > > type resolved this. Now the 0.95 and trunk builds fail for what
>> > looks
>> > > > > like
>> > > > > > a legitimate problem with a hanging test.
>> > > > > >
>> > > > >
>> > > > > --
>> > > > > Best regards,
>> > > > >
>> > > > >    - Andy
>> > > > >
>> > > > > Problems worthy of attack prove their worth by hitting back. -
>> Piet
>> > > Hein
>> > > > > (via Tom White)
>> > > > >
>> > > >
>> > >
>> > >
>> > >
>> > > --
>> > > Best regards,
>> > >
>> > >    - Andy
>> > >
>> > > Problems worthy of attack prove their worth by hitting back. - Piet
>> Hein
>> > > (via Tom White)
>> > >
>> >
>>
>>
>>
>> --
>> Best regards,
>>
>>    - Andy
>>
>> Problems worthy of attack prove their worth by hitting back. - Piet Hein
>> (via Tom White)
>>
>
>

This is a bug in the EC2 module for Jenkins. There are other bugs which
this one fixes so it's not a big deal relative to those. You have an
account on this system. You can easily go on and delete the slaves which
end up in offline state.
On Fri, Apr 5, 2013 at 4:39 PM, Ted Yu <[EMAIL PROTECTED]> wrote:

> Looks like 4 ECs Jenkins slaves are offline at the moment ...
>
>
> On Wed, Mar 27, 2013 at 1:19 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
> > Looks like Apache Jenkins went off several times this week.
> >
> > Is it difficult to hook up patching test with the new Jenkins ?
> >
> > Thanks
> >
> >
> > On Wed, Mar 27, 2013 at 7:49 AM, Andrew Purtell <[EMAIL PROTECTED]
> >wrote:
> >
> >> True, but unlike 0.94 the state of 0.95 and trunk is impacted by Stack's
> >> wrangling with Maven to find a sane site and assembly, a number of build
> >> failures are due to that. Also you'll note that prior to yesterday the
> >> Linux OOM killer was nuking the bloated Maven processes on the build
> >> slaves. Let's give these builds a bit of time for this stuff to get
> sorted
> >> out. The failures in 0.94 seem immediately actionable.
> >>
> >>
> >> On Wed, Mar 27, 2013 at 3:38 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
> >>
> >> > Trunk and 0.95 builds are not in good shape.
> >> > 0.95 builds have been failing for 32 times.
> >> >
> >> > On Apache Jenkins, looks like TestAssignmentManagerOnCluster has
> failed
> >> > quite often for 0.95 and trunk builds.
> >> >
> >> > On Wed, Mar 27, 2013 at 7:18 AM, Andrew Purtell <[EMAIL PROTECTED]>
> >> > wrote:
> >> >
> >> > > In general moving from using the m1.large (2 vcores, 7.5 GB RAM) to
> >> the
> >> > > m1.xlarge (4 vcores, 15 GB RAM) instance type for the slaves helped
> >> with
> >> > a
> >> > > build/test timeout, so now I'd about claim the test environment is
> >> sane.
> >> > We
> >> > > are now seeing that replication tests are flapping, occasionally
> >> timing
> >> > out
> >> > > internally:
> >> > >
> >> > > See
> >> > >
> >> > >
> >> >
> >>
> http://54.241.6.143/job/HBase-0.94/org.apache.hbase$hbase/24/testReport/junit/org.apache.hadoop.hbase.replication/TestReplicationQueueFailoverCompressed/queueFailover/
> >> > >
> >> > >
> >> > > and
> >> > >
> >> > >
> >> >
> >>
> http://54.241.6.143/job/HBase-0.94-Security/org.apache.hbase$hbase/7/testReport/junit/org.apache.hadoop.hbase.replication/TestReplicationQueueFailover/queueFailover/
> >> > >
> >> > >
> >> > > The 0.94 and 0.94-security builds are alternating between green and
> >> red
> >> > as
> >> > > a result.
> >> > >
> >> > > Perhaps we should reopen/revisit either adjusting the internal
> >> timeouts
> >> > for
> >> > > these tests or the other JIRA about moving minicluster replication
> >> tests
> >> > to
> >> > > hbase-it.
> >> > >
> >> > >
> >> > > On Wed, Mar 27, 2013 at 1:49 AM, Nick Dimiduk <[EMAIL PROTECTED]>
> >> > wrote:
> >> > >
> >> > > > On Tue, Mar 26, 2013 at 1:28 PM, Andrew Purtell <
> >> [EMAIL PROTECTED]>
> >> > > > wrote:
> >> > > >
> >> > > > > The HBase 0.94 build is now testing green!
> >> > > > > http://54.241.6.143/job/HBase-0.94/
> >> > > > >
> >> > > >
> >> > > > ^5!
> >> > > >
> >> > > > On Tue, Mar 26, 2013 at 1:47 AM, Andrew Purtell <
> >> [EMAIL PROTECTED]>
> >> > > > wrote:
> >> > > > >
> >> > > > > > I found that Maven was being killed on the slaves by the Linux
> >> OOM
> >> > > > killer
> >> > > > > > sometimes for >= 0.95. Seems the m1.large process didn't have
> >> > enough
> >> > > > > memory
> >> > > > > > to host the Jenkins slave, Maven with its 3G+ heap, and the
> >> forked
> >> > > JVMs
> >> > > > > for
> >> > > > > > the medium and large tests at the same time. Switching to the
> >> > > m1.xlarge
> >> > > > > > type resolved this. Now the 0.95 and trunk builds fail for
> what
> >> > looks
> >> > > > > like
> >> > > > > > a legitimate problem with a hanging test.
> >> > > > > >
> >> > > > >
> >> > > > > --
> >> > > > > Best regards,
> >> > > > >
> >> > > > >    - Andy
> >> > >
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Also, be careful to differentiate between slaves that are "offline" because
they are in the process of being launched, and those that are offline
because of that bug I mention. (It doesn't happen often but does happen.)
If you kill an "offline" slave being launched, this will just cause churn.
And if this seems like something you don't want to bother with, then just
don't worry about it.

On Fri, Apr 5, 2013 at 4:44 PM, Andrew Purtell <[EMAIL PROTECTED]> wrote:

> This is a bug in the EC2 module for Jenkins. There are other bugs which
> this one fixes so it's not a big deal relative to those. You have an
> account on this system. You can easily go on and delete the slaves which
> end up in offline state.
>
>
> On Fri, Apr 5, 2013 at 4:39 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
>> Looks like 4 ECs Jenkins slaves are offline at the moment ...
>>
>>
>> On Wed, Mar 27, 2013 at 1:19 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>>
>> > Looks like Apache Jenkins went off several times this week.
>> >
>> > Is it difficult to hook up patching test with the new Jenkins ?
>> >
>> > Thanks
>> >
>> >
>> > On Wed, Mar 27, 2013 at 7:49 AM, Andrew Purtell <[EMAIL PROTECTED]
>> >wrote:
>> >
>> >> True, but unlike 0.94 the state of 0.95 and trunk is impacted by
>> Stack's
>> >> wrangling with Maven to find a sane site and assembly, a number of
>> build
>> >> failures are due to that. Also you'll note that prior to yesterday the
>> >> Linux OOM killer was nuking the bloated Maven processes on the build
>> >> slaves. Let's give these builds a bit of time for this stuff to get
>> sorted
>> >> out. The failures in 0.94 seem immediately actionable.
>> >>
>> >>
>> >> On Wed, Mar 27, 2013 at 3:38 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>> >>
>> >> > Trunk and 0.95 builds are not in good shape.
>> >> > 0.95 builds have been failing for 32 times.
>> >> >
>> >> > On Apache Jenkins, looks like TestAssignmentManagerOnCluster has
>> failed
>> >> > quite often for 0.95 and trunk builds.
>> >> >
>> >> > On Wed, Mar 27, 2013 at 7:18 AM, Andrew Purtell <[EMAIL PROTECTED]
>> >
>> >> > wrote:
>> >> >
>> >> > > In general moving from using the m1.large (2 vcores, 7.5 GB RAM) to
>> >> the
>> >> > > m1.xlarge (4 vcores, 15 GB RAM) instance type for the slaves helped
>> >> with
>> >> > a
>> >> > > build/test timeout, so now I'd about claim the test environment is
>> >> sane.
>> >> > We
>> >> > > are now seeing that replication tests are flapping, occasionally
>> >> timing
>> >> > out
>> >> > > internally:
>> >> > >
>> >> > > See
>> >> > >
>> >> > >
>> >> >
>> >>
>> http://54.241.6.143/job/HBase-0.94/org.apache.hbase$hbase/24/testReport/junit/org.apache.hadoop.hbase.replication/TestReplicationQueueFailoverCompressed/queueFailover/
>> >> > >
>> >> > >
>> >> > > and
>> >> > >
>> >> > >
>> >> >
>> >>
>> http://54.241.6.143/job/HBase-0.94-Security/org.apache.hbase$hbase/7/testReport/junit/org.apache.hadoop.hbase.replication/TestReplicationQueueFailover/queueFailover/
>> >> > >
>> >> > >
>> >> > > The 0.94 and 0.94-security builds are alternating between green and
>> >> red
>> >> > as
>> >> > > a result.
>> >> > >
>> >> > > Perhaps we should reopen/revisit either adjusting the internal
>> >> timeouts
>> >> > for
>> >> > > these tests or the other JIRA about moving minicluster replication
>> >> tests
>> >> > to
>> >> > > hbase-it.
>> >> > >
>> >> > >
>> >> > > On Wed, Mar 27, 2013 at 1:49 AM, Nick Dimiduk <[EMAIL PROTECTED]>
>> >> > wrote:
>> >> > >
>> >> > > > On Tue, Mar 26, 2013 at 1:28 PM, Andrew Purtell <
>> >> [EMAIL PROTECTED]>
>> >> > > > wrote:
>> >> > > >
>> >> > > > > The HBase 0.94 build is now testing green!
>> >> > > > > http://54.241.6.143/job/HBase-0.94/
>> >> > > > >
>> >> > > >
>> >> > > > ^5!
>> >> > > >
>> >> > > > On Tue, Mar 26, 2013 at 1:47 AM, Andrew Purtell <
>> >> [EMAIL PROTECTED]>
>> >> > > > wrote:
>> >> > > > >
>> >> > > > > > I found that Maven was being killed on the slaves by the
>> Linux
>> >> OOM
>> >> > > > killer
>> >> > > > >
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Today I updated the EC2 plugin to see if that resolves the issue with
occasional failed slave launches. It looks good so far.
On Thu, Mar 21, 2013 at 9:38 PM, Stack <[EMAIL PROTECTED]> wrote:

> This machine is now running builds: http://54.241.6.143/    After some
> messing to fix memory allocation, it seems tests are failing legitimately
> now for both 0.94 and trunk.   They warrant fixing (thanks for setting this
> up Andrew).
> St.Ack
>
>
> On Tue, Mar 19, 2013 at 10:48 AM, Andrew Purtell <[EMAIL PROTECTED]
> >wrote:
>
> > I have checked with the folks here and until/unless the situation with
> ASF
> > Jenkins VMs improves, we can provide a hosted Jenkins up in EC2, in
> > us-west-1:
> >
> >   - A m1.large instance running 24/7 hosting Jenkins
> >
> >   - A pool of 5 m1.large instances serving as Jenkins slaves, managed by
> > Jenkins with its EC2 plugin:
> > https://wiki.jenkins-ci.org/display/JENKINS/Amazon+EC2+Plugin . These
> > would
> > be launched on demand depending on how many concurrent executors are
> needed
> > (we'd set up a 1 executor to 1 instance mapping) and reaped when
> inactive.
> > Jenkins' EC2 plugin handles those details.
> >
> > It would remain to be seen if this is better than the ASF Jenkins VMs,
> but
> > if the community is willing to try it, we'd be willing to set it up and
> > give out access. With EC2's IAM (http://aws.amazon.com/iam/) it's
> possible
> > to give all committers accounts for managing these resources. An m1.large
> > has:
> >     - 7.5 GiB memory
> >     - 4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units each)
> >     - 850 GB instance storage
> >     - 64-bit platform
> >     - I/O Performance: Moderate
> >
> >
> > On Tue, Mar 19, 2013 at 5:23 PM, Nick Dimiduk <[EMAIL PROTECTED]>
> wrote:
> >
> > > What, like travis-ci.org ?
> > >
> > > On Tue, Mar 19, 2013 at 8:49 AM, Andrew Purtell <[EMAIL PROTECTED]>
> > > wrote:
> > >
> > > > Would the HBase dev community be open to alternate hosted Jenkins
> test
> > > > infrastructure?
> > > >
> > > > --
> > > > Best regards,
> > > >
> > > >    - Andy
> > > >
> > > > Problems worthy of attack prove their worth by hitting back. - Piet
> > Hein
> > > > (via Tom White)
> > > >
> > >
> >
> >
> >
> > --
> > Best regards,
> >
> >    - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
> >
>

--
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Andy:
Have you noticed the following from recent EC2 trunk builds ?

cause : SiteToolException: The site descriptor cannot be resolved from the
repository: ArtifactResolutionException: Unable to locate site descriptor:
Could not transfer artifact org.apache:apache:xml:site_en:12 from/to
repository.jboss.org (
http://repository.jboss.org/nexus/content/groups/public-jboss/): Failed to
transfer file:
http://repository.jboss.org/nexus/content/groups/public-jboss/org/apache/apache/12/apache-12-site_en.xml.
Return code is: 503, ReasonPhrase:Service Temporarily Unavailable.
  org.apache:apache:xml:12

On Thu, Apr 18, 2013 at 12:46 PM, Andrew Purtell <[EMAIL PROTECTED]>wrote:

> Today I updated the EC2 plugin to see if that resolves the issue with
> occasional failed slave launches. It looks good so far.
>
>
> On Thu, Mar 21, 2013 at 9:38 PM, Stack <[EMAIL PROTECTED]> wrote:
>
> > This machine is now running builds: http://54.241.6.143/    After some
> > messing to fix memory allocation, it seems tests are failing legitimately
> > now for both 0.94 and trunk.   They warrant fixing (thanks for setting
> this
> > up Andrew).
> > St.Ack
> >
> >
> > On Tue, Mar 19, 2013 at 10:48 AM, Andrew Purtell <[EMAIL PROTECTED]
> > >wrote:
> >
> > > I have checked with the folks here and until/unless the situation with
> > ASF
> > > Jenkins VMs improves, we can provide a hosted Jenkins up in EC2, in
> > > us-west-1:
> > >
> > >   - A m1.large instance running 24/7 hosting Jenkins
> > >
> > >   - A pool of 5 m1.large instances serving as Jenkins slaves, managed
> by
> > > Jenkins with its EC2 plugin:
> > > https://wiki.jenkins-ci.org/display/JENKINS/Amazon+EC2+Plugin . These
> > > would
> > > be launched on demand depending on how many concurrent executors are
> > needed
> > > (we'd set up a 1 executor to 1 instance mapping) and reaped when
> > inactive.
> > > Jenkins' EC2 plugin handles those details.
> > >
> > > It would remain to be seen if this is better than the ASF Jenkins VMs,
> > but
> > > if the community is willing to try it, we'd be willing to set it up and
> > > give out access. With EC2's IAM (http://aws.amazon.com/iam/) it's
> > possible
> > > to give all committers accounts for managing these resources. An
> m1.large
> > > has:
> > >     - 7.5 GiB memory
> > >     - 4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units
> each)
> > >     - 850 GB instance storage
> > >     - 64-bit platform
> > >     - I/O Performance: Moderate
> > >
> > >
> > > On Tue, Mar 19, 2013 at 5:23 PM, Nick Dimiduk <[EMAIL PROTECTED]>
> > wrote:
> > >
> > > > What, like travis-ci.org ?
> > > >
> > > > On Tue, Mar 19, 2013 at 8:49 AM, Andrew Purtell <[EMAIL PROTECTED]
> >
> > > > wrote:
> > > >
> > > > > Would the HBase dev community be open to alternate hosted Jenkins
> > test
> > > > > infrastructure?
> > > > >
> > > > > --
> > > > > Best regards,
> > > > >
> > > > >    - Andy
> > > > >
> > > > > Problems worthy of attack prove their worth by hitting back. - Piet
> > > Hein
> > > > > (via Tom White)
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Best regards,
> > >
> > >    - Andy
> > >
> > > Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> > > (via Tom White)
> > >
> >
>
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>

Hi Ted,

Why are you asking me this? Do you think this is a problem with EC2
Jenkins?  Did you check repository.jboss.org ? Does it work for you?  This
is what I get at
http://repository.jboss.org/nexus/content/groups/public-jboss/org/apache/apache/12/apache-12-site_en.xml
 :

Service Temporarily Unavailable

The server is temporarily unable to service your request due to maintenance
downtime or capacity problems. Please try again later.
------------------------------
Apache Server at repository.jboss.org Port 80
On Thu, Apr 18, 2013 at 5:22 PM, Ted Yu <[EMAIL PROTECTED]> wrote:

> Andy:
> Have you noticed the following from recent EC2 trunk builds ?
>
> cause : SiteToolException: The site descriptor cannot be resolved from the
> repository: ArtifactResolutionException: Unable to locate site descriptor:
> Could not transfer artifact org.apache:apache:xml:site_en:12 from/to
> repository.jboss.org (
> http://repository.jboss.org/nexus/content/groups/public-jboss/): Failed to
> transfer file:
>
> http://repository.jboss.org/nexus/content/groups/public-jboss/org/apache/apache/12/apache-12-site_en.xml
> .
> Return code is: 503, ReasonPhrase:Service Temporarily Unavailable.
>   org.apache:apache:xml:12
>
> On Thu, Apr 18, 2013 at 12:46 PM, Andrew Purtell <[EMAIL PROTECTED]
> >wrote:
>
> > Today I updated the EC2 plugin to see if that resolves the issue with
> > occasional failed slave launches. It looks good so far.
> >
> >
> > On Thu, Mar 21, 2013 at 9:38 PM, Stack <[EMAIL PROTECTED]> wrote:
> >
> > > This machine is now running builds: http://54.241.6.143/    After some
> > > messing to fix memory allocation, it seems tests are failing
> legitimately
> > > now for both 0.94 and trunk.   They warrant fixing (thanks for setting
> > this
> > > up Andrew).
> > > St.Ack
> > >
> > >
> > > On Tue, Mar 19, 2013 at 10:48 AM, Andrew Purtell <[EMAIL PROTECTED]
> > > >wrote:
> > >
> > > > I have checked with the folks here and until/unless the situation
> with
> > > ASF
> > > > Jenkins VMs improves, we can provide a hosted Jenkins up in EC2, in
> > > > us-west-1:
> > > >
> > > >   - A m1.large instance running 24/7 hosting Jenkins
> > > >
> > > >   - A pool of 5 m1.large instances serving as Jenkins slaves, managed
> > by
> > > > Jenkins with its EC2 plugin:
> > > > https://wiki.jenkins-ci.org/display/JENKINS/Amazon+EC2+Plugin .
> These
> > > > would
> > > > be launched on demand depending on how many concurrent executors are
> > > needed
> > > > (we'd set up a 1 executor to 1 instance mapping) and reaped when
> > > inactive.
> > > > Jenkins' EC2 plugin handles those details.
> > > >
> > > > It would remain to be seen if this is better than the ASF Jenkins
> VMs,
> > > but
> > > > if the community is willing to try it, we'd be willing to set it up
> and
> > > > give out access. With EC2's IAM (http://aws.amazon.com/iam/) it's
> > > possible
> > > > to give all committers accounts for managing these resources. An
> > m1.large
> > > > has:
> > > >     - 7.5 GiB memory
> > > >     - 4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units
> > each)
> > > >     - 850 GB instance storage
> > > >     - 64-bit platform
> > > >     - I/O Performance: Moderate
> > > >
> > > >
> > > > On Tue, Mar 19, 2013 at 5:23 PM, Nick Dimiduk <[EMAIL PROTECTED]>
> > > wrote:
> > > >
> > > > > What, like travis-ci.org ?
> > > > >
> > > > > On Tue, Mar 19, 2013 at 8:49 AM, Andrew Purtell <
> [EMAIL PROTECTED]
> > >
> > > > > wrote:
> > > > >
> > > > > > Would the HBase dev community be open to alternate hosted Jenkins
> > > test
> > > > > > infrastructure?
> > > > > >
> > > > > > --
> > > > > > Best regards,
> > > > > >
> > > > > >    - Andy
> > > > > >
> > > > > > Problems worthy of attack prove their worth by hitting back. -
> Piet
> > > > Hein
> > > > > > (via Tom White)
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Best regards,
> > > >
> > > >    - Andy
> > > >
> > > > Problems worthy of attack prove their worth by hitting back. - Piet

Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Looks like http://repository.jboss.org/nexus/content/groups/public-jboss is
having problems.

I will just wait for it to come back.

On Thu, Apr 18, 2013 at 5:25 PM, Andrew Purtell <[EMAIL PROTECTED]> wrote:

> Hi Ted,
>
> Why are you asking me this? Do you think this is a problem with EC2
> Jenkins?  Did you check repository.jboss.org ? Does it work for you?  This
> is what I get at
>
> http://repository.jboss.org/nexus/content/groups/public-jboss/org/apache/apache/12/apache-12-site_en.xml
>  :
>
> Service Temporarily Unavailable
>
> The server is temporarily unable to service your request due to maintenance
> downtime or capacity problems. Please try again later.
> ------------------------------
> Apache Server at repository.jboss.org Port 80
>
>
>
>
> On Thu, Apr 18, 2013 at 5:22 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
> > Andy:
> > Have you noticed the following from recent EC2 trunk builds ?
> >
> > cause : SiteToolException: The site descriptor cannot be resolved from
> the
> > repository: ArtifactResolutionException: Unable to locate site
> descriptor:
> > Could not transfer artifact org.apache:apache:xml:site_en:12 from/to
> > repository.jboss.org (
> > http://repository.jboss.org/nexus/content/groups/public-jboss/): Failed
> to
> > transfer file:
> >
> >
> http://repository.jboss.org/nexus/content/groups/public-jboss/org/apache/apache/12/apache-12-site_en.xml
> > .
> > Return code is: 503, ReasonPhrase:Service Temporarily Unavailable.
> >   org.apache:apache:xml:12
> >
> > On Thu, Apr 18, 2013 at 12:46 PM, Andrew Purtell <[EMAIL PROTECTED]
> > >wrote:
> >
> > > Today I updated the EC2 plugin to see if that resolves the issue with
> > > occasional failed slave launches. It looks good so far.
> > >
> > >
> > > On Thu, Mar 21, 2013 at 9:38 PM, Stack <[EMAIL PROTECTED]> wrote:
> > >
> > > > This machine is now running builds: http://54.241.6.143/    After
> some
> > > > messing to fix memory allocation, it seems tests are failing
> > legitimately
> > > > now for both 0.94 and trunk.   They warrant fixing (thanks for
> setting
> > > this
> > > > up Andrew).
> > > > St.Ack
> > > >
> > > >
> > > > On Tue, Mar 19, 2013 at 10:48 AM, Andrew Purtell <
> [EMAIL PROTECTED]
> > > > >wrote:
> > > >
> > > > > I have checked with the folks here and until/unless the situation
> > with
> > > > ASF
> > > > > Jenkins VMs improves, we can provide a hosted Jenkins up in EC2, in
> > > > > us-west-1:
> > > > >
> > > > >   - A m1.large instance running 24/7 hosting Jenkins
> > > > >
> > > > >   - A pool of 5 m1.large instances serving as Jenkins slaves,
> managed
> > > by
> > > > > Jenkins with its EC2 plugin:
> > > > > https://wiki.jenkins-ci.org/display/JENKINS/Amazon+EC2+Plugin .
> > These
> > > > > would
> > > > > be launched on demand depending on how many concurrent executors
> are
> > > > needed
> > > > > (we'd set up a 1 executor to 1 instance mapping) and reaped when
> > > > inactive.
> > > > > Jenkins' EC2 plugin handles those details.
> > > > >
> > > > > It would remain to be seen if this is better than the ASF Jenkins
> > VMs,
> > > > but
> > > > > if the community is willing to try it, we'd be willing to set it up
> > and
> > > > > give out access. With EC2's IAM (http://aws.amazon.com/iam/) it's
> > > > possible
> > > > > to give all committers accounts for managing these resources. An
> > > m1.large
> > > > > has:
> > > > >     - 7.5 GiB memory
> > > > >     - 4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units
> > > each)
> > > > >     - 850 GB instance storage
> > > > >     - 64-bit platform
> > > > >     - I/O Performance: Moderate
> > > > >
> > > > >
> > > > > On Tue, Mar 19, 2013 at 5:23 PM, Nick Dimiduk <[EMAIL PROTECTED]>
> > > > wrote:
> > > > >
> > > > > > What, like travis-ci.org ?
> > > > > >
> > > > > > On Tue, Mar 19, 2013 at 8:49 AM, Andrew Purtell <
> > [EMAIL PROTECTED]
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Would the HBase dev community be open to alternate hosted
> Jenkins
> >

This is the second time you have pointed out a non-problem with EC2
Jenkins. Please do some basic sanity checking first.

Also, you have not made any positive comments or contributions here, only
pointed out what you believe are problems with my volunteer effort here.
You may not intend it to be like this, but it comes across as criticism. I
don't expect any kind of "thank you" but this is more like "fuck you", so
please bear that in mind.
On Thu, Apr 18, 2013 at 5:29 PM, Ted Yu <[EMAIL PROTECTED]> wrote:

> Looks like http://repository.jboss.org/nexus/content/groups/public-jbossis
> having problems.
>
> I will just wait for it to come back.
>
> On Thu, Apr 18, 2013 at 5:25 PM, Andrew Purtell <[EMAIL PROTECTED]>
> wrote:
>
> > Hi Ted,
> >
> > Why are you asking me this? Do you think this is a problem with EC2
> > Jenkins?  Did you check repository.jboss.org ? Does it work for you?
>  This
> > is what I get at
> >
> >
> http://repository.jboss.org/nexus/content/groups/public-jboss/org/apache/apache/12/apache-12-site_en.xml
> >  :
> >
> > Service Temporarily Unavailable
> >
> > The server is temporarily unable to service your request due to
> maintenance
> > downtime or capacity problems. Please try again later.
> > ------------------------------
> > Apache Server at repository.jboss.org Port 80
> >
> >
> >
> >
> > On Thu, Apr 18, 2013 at 5:22 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
> >
> > > Andy:
> > > Have you noticed the following from recent EC2 trunk builds ?
> > >
> > > cause : SiteToolException: The site descriptor cannot be resolved from
> > the
> > > repository: ArtifactResolutionException: Unable to locate site
> > descriptor:
> > > Could not transfer artifact org.apache:apache:xml:site_en:12 from/to
> > > repository.jboss.org (
> > > http://repository.jboss.org/nexus/content/groups/public-jboss/):
> Failed
> > to
> > > transfer file:
> > >
> > >
> >
> http://repository.jboss.org/nexus/content/groups/public-jboss/org/apache/apache/12/apache-12-site_en.xml
> > > .
> > > Return code is: 503, ReasonPhrase:Service Temporarily Unavailable.
> > >   org.apache:apache:xml:12
> > >
> > > On Thu, Apr 18, 2013 at 12:46 PM, Andrew Purtell <[EMAIL PROTECTED]
> > > >wrote:
> > >
> > > > Today I updated the EC2 plugin to see if that resolves the issue with
> > > > occasional failed slave launches. It looks good so far.
> > > >
> > > >
> > > > On Thu, Mar 21, 2013 at 9:38 PM, Stack <[EMAIL PROTECTED]> wrote:
> > > >
> > > > > This machine is now running builds: http://54.241.6.143/    After
> > some
> > > > > messing to fix memory allocation, it seems tests are failing
> > > legitimately
> > > > > now for both 0.94 and trunk.   They warrant fixing (thanks for
> > setting
> > > > this
> > > > > up Andrew).
> > > > > St.Ack
> > > > >
> > > > >
> > > > > On Tue, Mar 19, 2013 at 10:48 AM, Andrew Purtell <
> > [EMAIL PROTECTED]
> > > > > >wrote:
> > > > >
> > > > > > I have checked with the folks here and until/unless the situation
> > > with
> > > > > ASF
> > > > > > Jenkins VMs improves, we can provide a hosted Jenkins up in EC2,
> in
> > > > > > us-west-1:
> > > > > >
> > > > > >   - A m1.large instance running 24/7 hosting Jenkins
> > > > > >
> > > > > >   - A pool of 5 m1.large instances serving as Jenkins slaves,
> > managed
> > > > by
> > > > > > Jenkins with its EC2 plugin:
> > > > > > https://wiki.jenkins-ci.org/display/JENKINS/Amazon+EC2+Plugin .
> > > These
> > > > > > would
> > > > > > be launched on demand depending on how many concurrent executors
> > are
> > > > > needed
> > > > > > (we'd set up a 1 executor to 1 instance mapping) and reaped when
> > > > > inactive.
> > > > > > Jenkins' EC2 plugin handles those details.
> > > > > >
> > > > > > It would remain to be seen if this is better than the ASF Jenkins
> > > VMs,
> > > > > but
> > > > > > if the community is willing to try it, we'd be willing to set it
> up
> > > and
> > > > > > give out access. With EC2's IAM (http://aws.amazon.com/iam/)
> it's
> > > > > possible

Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Since the situation with ASF Jenkins seems better (please correct me if
wrong), we are going to scale back HBase jobs on the Jenkins at
http://54.241.6.143/ to one execution per day. Please let me know if you
have any concerns.
On Tue, Mar 19, 2013 at 10:48 AM, Andrew Purtell <[EMAIL PROTECTED]>wrote:

> I have checked with the folks here and until/unless the situation with ASF
> Jenkins VMs improves, we can provide a hosted Jenkins up in EC2, in
> us-west-1:
>
>   - A m1.large instance running 24/7 hosting Jenkins
>
>   - A pool of 5 m1.large instances serving as Jenkins slaves, managed by
> Jenkins with its EC2 plugin:
> https://wiki.jenkins-ci.org/display/JENKINS/Amazon+EC2+Plugin . These
> would be launched on demand depending on how many concurrent executors are
> needed (we'd set up a 1 executor to 1 instance mapping) and reaped when
> inactive. Jenkins' EC2 plugin handles those details.
>
> It would remain to be seen if this is better than the ASF Jenkins VMs, but
> if the community is willing to try it, we'd be willing to set it up and
> give out access. With EC2's IAM (http://aws.amazon.com/iam/) it's
> possible to give all committers accounts for managing these resources. An
> m1.large has:
>     - 7.5 GiB memory
>     - 4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units each)
>     - 850 GB instance storage
>     - 64-bit platform
>     - I/O Performance: Moderate
>
>
>  On Tue, Mar 19, 2013 at 5:23 PM, Nick Dimiduk <[EMAIL PROTECTED]> wrote:
>
>> What, like travis-ci.org ?
>>
>> On Tue, Mar 19, 2013 at 8:49 AM, Andrew Purtell <[EMAIL PROTECTED]>
>> wrote:
>>
>> > Would the HBase dev community be open to alternate hosted Jenkins test
>> > infrastructure?
>> >
>> > --
>> > Best regards,
>> >
>> >    - Andy
>> >
>> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
>> > (via Tom White)
>> >
>>
>
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>

--
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Thanks for putting so much into that to get us past that rough spot.
On Mon, Jun 10, 2013 at 11:14 AM, Andrew Purtell <[EMAIL PROTECTED]>wrote:

> Since the situation with ASF Jenkins seems better (please correct me if
> wrong), we are going to scale back HBase jobs on the Jenkins at
> http://54.241.6.143/ to one execution per day. Please let me know if you
> have any concerns.
>
>
> On Tue, Mar 19, 2013 at 10:48 AM, Andrew Purtell <[EMAIL PROTECTED]
> >wrote:
>
> > I have checked with the folks here and until/unless the situation with
> ASF
> > Jenkins VMs improves, we can provide a hosted Jenkins up in EC2, in
> > us-west-1:
> >
> >   - A m1.large instance running 24/7 hosting Jenkins
> >
> >   - A pool of 5 m1.large instances serving as Jenkins slaves, managed by
> > Jenkins with its EC2 plugin:
> > https://wiki.jenkins-ci.org/display/JENKINS/Amazon+EC2+Plugin . These
> > would be launched on demand depending on how many concurrent executors
> are
> > needed (we'd set up a 1 executor to 1 instance mapping) and reaped when
> > inactive. Jenkins' EC2 plugin handles those details.
> >
> > It would remain to be seen if this is better than the ASF Jenkins VMs,
> but
> > if the community is willing to try it, we'd be willing to set it up and
> > give out access. With EC2's IAM (http://aws.amazon.com/iam/) it's
> > possible to give all committers accounts for managing these resources. An
> > m1.large has:
> >     - 7.5 GiB memory
> >     - 4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units each)
> >     - 850 GB instance storage
> >     - 64-bit platform
> >     - I/O Performance: Moderate
> >
> >
> >  On Tue, Mar 19, 2013 at 5:23 PM, Nick Dimiduk <[EMAIL PROTECTED]>
> wrote:
> >
> >> What, like travis-ci.org ?
> >>
> >> On Tue, Mar 19, 2013 at 8:49 AM, Andrew Purtell <[EMAIL PROTECTED]>
> >> wrote:
> >>
> >> > Would the HBase dev community be open to alternate hosted Jenkins test
> >> > infrastructure?
> >> >
> >> > --
> >> > Best regards,
> >> >
> >> >    - Andy
> >> >
> >> > Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> >> > (via Tom White)
> >> >
> >>
> >
> >
> >
> > --
> > Best regards,
> >
> >    - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
> >
>
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>

Thanks Elliott.

This reminds me... to update, we've retained the SCM polling and daily
builds for 0.94 and 0.94-security. All of the other builds are set to
execute once per day at a random time.
On Mon, Jun 10, 2013 at 12:42 PM, Elliott Clark <[EMAIL PROTECTED]> wrote:

> Thanks for putting so much into that to get us past that rough spot.
>
>
> On Mon, Jun 10, 2013 at 11:14 AM, Andrew Purtell <[EMAIL PROTECTED]
> >wrote:
>
> > Since the situation with ASF Jenkins seems better (please correct me if
> > wrong), we are going to scale back HBase jobs on the Jenkins at
> > http://54.241.6.143/ to one execution per day. Please let me know if you
> > have any concerns.
> >
> >
> > On Tue, Mar 19, 2013 at 10:48 AM, Andrew Purtell <[EMAIL PROTECTED]
> > >wrote:
> >
> > > I have checked with the folks here and until/unless the situation with
> > ASF
> > > Jenkins VMs improves, we can provide a hosted Jenkins up in EC2, in
> > > us-west-1:
> > >
> > >   - A m1.large instance running 24/7 hosting Jenkins
> > >
> > >   - A pool of 5 m1.large instances serving as Jenkins slaves, managed
> by
> > > Jenkins with its EC2 plugin:
> > > https://wiki.jenkins-ci.org/display/JENKINS/Amazon+EC2+Plugin . These
> > > would be launched on demand depending on how many concurrent executors
> > are
> > > needed (we'd set up a 1 executor to 1 instance mapping) and reaped when
> > > inactive. Jenkins' EC2 plugin handles those details.
> > >
> > > It would remain to be seen if this is better than the ASF Jenkins VMs,
> > but
> > > if the community is willing to try it, we'd be willing to set it up and
> > > give out access. With EC2's IAM (http://aws.amazon.com/iam/) it's
> > > possible to give all committers accounts for managing these resources.
> An
> > > m1.large has:
> > >     - 7.5 GiB memory
> > >     - 4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units
> each)
> > >     - 850 GB instance storage
> > >     - 64-bit platform
> > >     - I/O Performance: Moderate
> > >
> > >
> > >  On Tue, Mar 19, 2013 at 5:23 PM, Nick Dimiduk <[EMAIL PROTECTED]>
> > wrote:
> > >
> > >> What, like travis-ci.org ?
> > >>
> > >> On Tue, Mar 19, 2013 at 8:49 AM, Andrew Purtell <[EMAIL PROTECTED]>
> > >> wrote:
> > >>
> > >> > Would the HBase dev community be open to alternate hosted Jenkins
> test
> > >> > infrastructure?
> > >> >
> > >> > --
> > >> > Best regards,
> > >> >
> > >> >    - Andy
> > >> >
> > >> > Problems worthy of attack prove their worth by hitting back. - Piet
> > Hein
> > >> > (via Tom White)
> > >> >
> > >>
> > >
> > >
> > >
> > > --
> > > Best regards,
> > >
> > >    - Andy
> > >
> > > Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> > > (via Tom White)
> > >
> >
> >
> >
> > --
> > Best regards,
> >
> >    - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
> >
>

--
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Sounds good Andrew.  Its been useful having another perspective.
On Mon, Jun 10, 2013 at 11:14 AM, Andrew Purtell <[EMAIL PROTECTED]>wrote:

> Since the situation with ASF Jenkins seems better (please correct me if
> wrong), we are going to scale back HBase jobs on the Jenkins at
> http://54.241.6.143/ to one execution per day. Please let me know if you
> have any concerns.
>
>
> On Tue, Mar 19, 2013 at 10:48 AM, Andrew Purtell <[EMAIL PROTECTED]
> >wrote:
>
> > I have checked with the folks here and until/unless the situation with
> ASF
> > Jenkins VMs improves, we can provide a hosted Jenkins up in EC2, in
> > us-west-1:
> >
> >   - A m1.large instance running 24/7 hosting Jenkins
> >
> >   - A pool of 5 m1.large instances serving as Jenkins slaves, managed by
> > Jenkins with its EC2 plugin:
> > https://wiki.jenkins-ci.org/display/JENKINS/Amazon+EC2+Plugin . These
> > would be launched on demand depending on how many concurrent executors
> are
> > needed (we'd set up a 1 executor to 1 instance mapping) and reaped when
> > inactive. Jenkins' EC2 plugin handles those details.
> >
> > It would remain to be seen if this is better than the ASF Jenkins VMs,
> but
> > if the community is willing to try it, we'd be willing to set it up and
> > give out access. With EC2's IAM (http://aws.amazon.com/iam/) it's
> > possible to give all committers accounts for managing these resources. An
> > m1.large has:
> >     - 7.5 GiB memory
> >     - 4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units each)
> >     - 850 GB instance storage
> >     - 64-bit platform
> >     - I/O Performance: Moderate
> >
> >
> >  On Tue, Mar 19, 2013 at 5:23 PM, Nick Dimiduk <[EMAIL PROTECTED]>
> wrote:
> >
> >> What, like travis-ci.org ?
> >>
> >> On Tue, Mar 19, 2013 at 8:49 AM, Andrew Purtell <[EMAIL PROTECTED]>
> >> wrote:
> >>
> >> > Would the HBase dev community be open to alternate hosted Jenkins test
> >> > infrastructure?
> >> >
> >> > --
> >> > Best regards,
> >> >
> >> >    - Andy
> >> >
> >> > Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> >> > (via Tom White)
> >> >
> >>
> >
> >
> >
> > --
> > Best regards,
> >
> >    - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
> >
>
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>