Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Bigtop >> mail # dev >> Re: permanent builds on jenkins


Copy link to this message
-
Re: permanent builds on jenkins
And also it is 'permanent'.

On Sun, Jul 07, 2013 at 02:37PM, Konstantin Boudnik wrote:
> [Moving user@ to Bcc: and cross-posting to dev@ This is a more dev discussion,
> let's keep it there]
>
> I think there's a confusion of some sort here. Apache normally doesn't provide
> binary releases. A release - in ASF - is a source code. While binaries can be
> provided as a convenience artifacts it is most often isn't a case.
>
> Now, a Bigtop release is a fixed BOM file that states what versions of what
> components were built and tested to work against each other. If you are
> looking for particular system packages - it is very easy to get by
>   make rpm|deb
> from a particular release tag. I believe we do publish binary packages for
> each release, but you should understand that these are stored on s3 that costs
> money. Right now, BT build infrastructure is running on the funds donated by
> commercial vendors (Cloudera is the only one so far, others are chiming in as
> we speak).
>
> Same goes for a VM: if you need a VM with particular release bits in it: you
> just check out the tag and run boxgrinder that will create identical VM each
> time you run the build. The VM in question isn't meant for development of BT.
> This is a mere test-bench for people who want to try the latest release of
> Hadoop based stack and have no skills of going through the whole process by
> themselves.
>
> I wish we could keep a copy of VMs for all stable builds, but this is a simple
> matter of the costs vs. convenience.
>
> As for #3 below: this is pretty much how our Jenkins infra works. There's a
> bunch of slave VMs that are used for package building/installation/testing. If
> you have some improvements in mind - feel free to open a JIRA tickets and
> contribute a patch/design document/etc. Any help is highly welcome.
>  
> Cos
>
> On Sun, Jul 07, 2013 at 12:03PM, Jay Vyas wrote:
> > Hi bruno:
> >
> > 1) I've noticed that https://builds.apache.org/view/A-F/view/Bigtop/ is not
> > valid but "https://builds.apache.org/job/Bigtop-trunk/" seems to work.  Is
> > this an error on the website?
> >
> > 2) http://bigtop01.cloudera.org:8080/job/Bigtop-VM-matrix/ Seems to build
> > direct from HEAD.
> >
> > Are there actual bigtop releases which aren't on jenkins?  Everything seems
> > to be centered around the build server, rather than actual frozen,
> > downloadable releases.
> >
> > 3) Regarding testing the VMs.  Maybe to start some simple virt-install
> > scripts could help confirm that the KVM builds, at least, are working.  If
> > we could get them working with static IPs then we could even clone down the
> > bigtop source code and run bigtop hadoop tests on the VMs after
> > construction as a validation step.
> >
> >
> >
> >
> > On Sat, Jul 6, 2013 at 8:58 PM, Bruno MahИ <[EMAIL PROTECTED]> wrote:
> >
> > > On 07/06/2013 07:33 AM, Jay Vyas wrote:
> > >
> > >> Hi bigtop:
> > >>
> > >> Are there any permanant builds saved on jenkins (for the VM matrix)?
> > >>
> > >> If not it would be nice to add them for certain known well tested,
> > >> working disk images .
> > >>
> > >> (for context, I'm currently running Mr2 build of the KVM box and it
> > >> appears to have some intermittent write issues on the DataNode path, and
> > >> also, my namenode appears to really like being in safe mode.  these
> > >> could just be due to VM setup though, as im changing some things like
> > >> adding static IPs and data node write paths... so nothing to be alarmed
> > >> about.)
> > >>
> > >> --
> > >> Jay Vyas
> > >> http://jayunit100.blogspot.com
> > >>
> > >
> > > Hi Jay,
> > >
> > > Could you defined "permanent build" ?
> > > I am not sure if this fits your requirement, but jenkins has a link to the
> > > latest successful build (ex: http://bigtop01.cloudera.org:**
> > > 8080/job/Bigtop-VM-matrix/BR=**master,KIND=kvm,label=**
> > > fedora16/lastSuccessfulBuild/**artifact/bigtop-vm-kvm-master.**tar.gz<http://bigtop01.cloudera.org:8080/job/Bigtop-VM-matrix/BR=master,KIND=kvm,label=fedora16/lastSuccessfulBuild/artifact/bigtop-vm-kvm-master.tar.gz>)