Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # dev - Hadoop build slaves software


Copy link to this message
-
Re: Hadoop build slaves software
Konstantin Boudnik 2013-01-04, 19:10
Do I hear puppet? :)

Cos

On Fri, Jan 04, 2013 at 11:08AM, Todd Lipcon wrote:
> I agree -- I'd like to see us have a shell script of some sort which,
> given a prefix, downloads and installs the needed toolchain
> dependencies.
>
> We could then download that script onto the build machines and install
> into something like /opt/hadoop-toolchain/
> AFAIK the only real dependencies we have where the Ubuntu packages are
> too old are protoc and maven, so shouldnt be too tough.
>
> -Todd
>
> On Fri, Jan 4, 2013 at 10:59 AM, Rajiv Chittajallu <[EMAIL PROTECTED]> wrote:
> > asf008 has been up for a while. It was probably just added as a slave.
> >
> > All the dependencies should probably be installed in a build_prefix, to
> > avoid conflict to OS specific packages and allows multiple projects to
> > build on the same machines. This is an better alternative to
> > provisioning vms for unique builds.
> >
> > -rajive
> >
> > Giridharan Kesavan wrote on 01/04/13 at 09:31:55 -0800:
> >>   Im on it
> >>
> >>   -Giri
> >>
> >>   On Thu, Jan 3, 2013 at 11:24 PM, Todd Lipcon <[1][EMAIL PROTECTED]> wrote:
> >>
> >>     Hey folks,
> >>
> >>     It looks like hadoop8 has recently come back online as a build slave,
> >>     but is failing all the builds because it has an ancient version of
> >>     protobuf (2.2.0):
> >>     todd@asf008:~$ protoc  --version
> >>     libprotoc 2.2.0
> >>
> >>     In contrast, other slaves have 2.4.1:
> >>     todd@asf001:~$ protoc --version
> >>     libprotoc 2.4.1
> >>
> >>     asf001 has the newer protoc in /usr/local/bin but asf008 does not.
> >>     Does anyone know how software is meant to be deployed on these build
> >>     slaves? I'm happy to download and install protobuf 2.4.1 into
> >>     /usr/local on asf008 if manual installation is the name of the game,
> >>     but it seems like we should be doing something a little more
> >>     reproducible than one-off builds by rando developers to manage our
> >>     toolchain on the Jenkins slaves.
> >>     -Todd
> >>     --
> >>     Todd Lipcon
> >>     Software Engineer, Cloudera
> >>
> >>References
> >>
> >>   Visible links
> >>   1. mailto:[EMAIL PROTECTED]
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera