Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # dev - The Jenkins VMs are increasingly slow / overloaded


+
Andrew Purtell 2013-03-19, 15:49
+
Nick Dimiduk 2013-03-19, 16:23
+
Andrew Purtell 2013-03-19, 17:48
+
Stack 2013-03-19, 19:00
+
Andrew Purtell 2013-03-19, 23:30
+
lars hofhansl 2013-03-20, 06:33
Copy link to this message
-
Re: The Jenkins VMs are increasingly slow / overloaded
Andrew Purtell 2013-03-20, 08:57
I have this almost ready for you to play around with. Aim is to get a clean
trunk and 0.94 build first before turning it loose. The build deps are all
set up in the custom AMI, ASF build tools, protoc, etc. At this point just
working on debugging the Jenkins SVN plug in. "svn: E155005: No write-lock"
WTF? I may have to switch to using Git.

On Wed, Mar 20, 2013 at 7:33 AM, lars hofhansl <[EMAIL PROTECTED]> wrote:

> The current 0.94 run on ubuntu1 has been running for > 80 minutes and has
> not run a single test, yet (still compiling). This is pretty terrible.
> Anything you can do will help :)
>
>
>
> ________________________________
>  From: Andrew Purtell <[EMAIL PROTECTED]>
> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
> Sent: Tuesday, March 19, 2013 4:30 PM
> Subject: Re: The Jenkins VMs are increasingly slow / overloaded
>
> I'm setting it up now. The Jenkins host is ready, working on the PV AMI for
> the slaves.
>
> > Should we try and organize a whiparound among orgs to help pay for it
> once its up and running?
>
> Sure, if we pool resources then we could pay for more parallel instances.
>
> We could also consider more interesting (and expensive.. $4/hr/instance)
> types for specialized tests: like hi1.4xlarge, which has 60 GB of RAM and 2
> TB of SSD backed storage, although they may not be available in us-west-1
> yet.
>
>
> On Tue, Mar 19, 2013 at 8:00 PM, Stack <[EMAIL PROTECTED]> wrote:
>
> > +1 if can be done.  Can point our CI link over there.  What help you need
> > Andrew?  Should we try and organize a whiparound among orgs to help pay
> for
> > it once its up and running?
> >
> > I've been on and off working w/ Roman to get our IT builds running over
> on
> > bigtop but currently stuck on this strange issue in maven where it
> compiles
> > using source and target of 1.4 on his laptop and on bigtop even though it
> > is taking 1.6 as source and target properties and even though if he cuts
> > and pastes the command mvn claims to be running into his terminal, it
> just
> > works......
> >
> > St.Ack
> >
> >
> > On Tue, Mar 19, 2013 at 10:48 AM, Andrew Purtell <[EMAIL PROTECTED]
> > >wrote:
> >
> > > I have checked with the folks here and until/unless the situation with
> > ASF
> > > Jenkins VMs improves, we can provide a hosted Jenkins up in EC2, in
> > > us-west-1:
> > >
> > >   - A m1.large instance running 24/7 hosting Jenkins
> > >
> > >   - A pool of 5 m1.large instances serving as Jenkins slaves, managed
> by
> > > Jenkins with its EC2 plugin:
> > > https://wiki.jenkins-ci.org/display/JENKINS/Amazon+EC2+Plugin . These
> > > would
> > > be launched on demand depending on how many concurrent executors are
> > needed
> > > (we'd set up a 1 executor to 1 instance mapping) and reaped when
> > inactive.
> > > Jenkins' EC2 plugin handles those details.
> > >
> > > It would remain to be seen if this is better than the ASF Jenkins VMs,
> > but
> > > if the community is willing to try it, we'd be willing to set it up and
> > > give out access. With EC2's IAM (http://aws.amazon.com/iam/) it's
> > possible
> > > to give all committers accounts for managing these resources. An
> m1.large
> > > has:
> > >     - 7.5 GiB memory
> > >     - 4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units
> each)
> > >     - 850 GB instance storage
> > >     - 64-bit platform
> > >     - I/O Performance: Moderate
> > >
> > >
> > > On Tue, Mar 19, 2013 at 5:23 PM, Nick Dimiduk <[EMAIL PROTECTED]>
> > wrote:
> > >
> > > > What, like travis-ci.org ?
> > > >
> > > > On Tue, Mar 19, 2013 at 8:49 AM, Andrew Purtell <[EMAIL PROTECTED]
> >
> > > > wrote:
> > > >
> > > > > Would the HBase dev community be open to alternate hosted Jenkins
> > test
> > > > > infrastructure?
> > > > >
> > > > > --
> > > > > Best regards,
> > > > >
> > > > >    - Andy
> > > > >
> > > > > Problems worthy of attack prove their worth by hitting back. - Piet
> > > Hein
> > > > > (via Tom White)
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Best regards,

Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
+
Andrew Purtell 2013-03-20, 21:37
+
Ted Yu 2013-03-20, 21:39
+
Andrew Purtell 2013-03-20, 21:44
+
Andrew Purtell 2013-03-20, 21:44
+
Nick Dimiduk 2013-03-19, 20:50
+
Stack 2013-03-22, 04:38
+
Stack 2013-03-23, 04:51
+
Andrew Purtell 2013-03-23, 10:11
+
Stack 2013-03-23, 20:24
+
Andrew Purtell 2013-03-24, 05:56
+
Andrew Purtell 2013-03-26, 00:47
+
Andrew Purtell 2013-03-26, 20:28
+
Nick Dimiduk 2013-03-27, 00:49
+
Andrew Purtell 2013-03-27, 14:18
+
Ted Yu 2013-03-27, 14:38
+
Andrew Purtell 2013-03-27, 14:49
+
Ted Yu 2013-03-27, 20:19
+
Ted Yu 2013-04-05, 23:39
+
Andrew Purtell 2013-04-05, 23:44
+
Andrew Purtell 2013-04-05, 23:48
+
Stack 2013-03-26, 21:59
+
Andrew Purtell 2013-03-26, 22:14
+
Andrew Purtell 2013-04-18, 19:46
+
Ted Yu 2013-04-19, 00:22
+
Andrew Purtell 2013-04-19, 00:25
+
Ted Yu 2013-04-19, 00:29
+
Andrew Purtell 2013-04-19, 00:34
+
Andrew Purtell 2013-06-10, 18:14
+
Stack 2013-06-11, 00:03
+
Elliott Clark 2013-06-10, 19:42
+
Andrew Purtell 2013-06-10, 20:50
+
Andrew Purtell 2013-07-15, 22:27
+
Ted Yu 2013-07-15, 22:30
+
Andrew Purtell 2013-07-15, 22:37