Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # dev - Hadoop build slaves software


Copy link to this message
-
Re: Hadoop build slaves software
Todd Lipcon 2013-01-04, 19:08
I agree -- I'd like to see us have a shell script of some sort which,
given a prefix, downloads and installs the needed toolchain
dependencies.

We could then download that script onto the build machines and install
into something like /opt/hadoop-toolchain/
AFAIK the only real dependencies we have where the Ubuntu packages are
too old are protoc and maven, so shouldnt be too tough.

-Todd

On Fri, Jan 4, 2013 at 10:59 AM, Rajiv Chittajallu <[EMAIL PROTECTED]> wrote:
> asf008 has been up for a while. It was probably just added as a slave.
>
> All the dependencies should probably be installed in a build_prefix, to
> avoid conflict to OS specific packages and allows multiple projects to
> build on the same machines. This is an better alternative to
> provisioning vms for unique builds.
>
> -rajive
>
> Giridharan Kesavan wrote on 01/04/13 at 09:31:55 -0800:
>>   Im on it
>>
>>   -Giri
>>
>>   On Thu, Jan 3, 2013 at 11:24 PM, Todd Lipcon <[1][EMAIL PROTECTED]> wrote:
>>
>>     Hey folks,
>>
>>     It looks like hadoop8 has recently come back online as a build slave,
>>     but is failing all the builds because it has an ancient version of
>>     protobuf (2.2.0):
>>     todd@asf008:~$ protoc  --version
>>     libprotoc 2.2.0
>>
>>     In contrast, other slaves have 2.4.1:
>>     todd@asf001:~$ protoc --version
>>     libprotoc 2.4.1
>>
>>     asf001 has the newer protoc in /usr/local/bin but asf008 does not.
>>     Does anyone know how software is meant to be deployed on these build
>>     slaves? I'm happy to download and install protobuf 2.4.1 into
>>     /usr/local on asf008 if manual installation is the name of the game,
>>     but it seems like we should be doing something a little more
>>     reproducible than one-off builds by rando developers to manage our
>>     toolchain on the Jenkins slaves.
>>     -Todd
>>     --
>>     Todd Lipcon
>>     Software Engineer, Cloudera
>>
>>References
>>
>>   Visible links
>>   1. mailto:[EMAIL PROTECTED]

--
Todd Lipcon
Software Engineer, Cloudera