I have been looking at BigTop and was trying to generate RPMs for a few Hadoop related projects. However, the RPMs generated are not relocatable. Just want to check if I am missing out on some configuration parameters for generating relocatable RPMs or this is not supported yet.
Like Peter, I believe this would be a substantial amount of work. Relocatable RPMs work well when you mainly need a bunch of files installed on the machine. However, Bigtop is a collection of interconnected components and files that integrate them with the service and configuration management of the host OS. That is not so easy to do in a relocatable manner. On Wed, Jul 9, 2014 at 2:45 PM, plinnell <[EMAIL PROTECTED]> wrote:
I wonder if relocatable DEBs are possible. (Some quick googling suggests not?) If not, relocatable RPMs would be a substantial amount of work for a half measure.
I also think that if looking for deployment vehicles supporting concurrent installation of multiple component versions, we'd be better served putting project energy into LXC based deployment management and packaging. (That could be _really_ interesting, if for example containers have a late binding on dependencies, where they ask other containers during boot and service discovery to supply them with packages to install... I know, a crazy idea, not meant to lead this discussion off on a tangent) On Wed, Jul 9, 2014 at 2:45 PM, Sean Mackrory <[EMAIL PROTECTED]> wrote:
Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
@Peter The relocatable RPMs are intended for installation rather than binding/deploying to containers, so need to be able to configure the installation directory.
Based on my understanding, BigTop makes the RPMs not relocatable because it deploys multiple Hadoop related projects, the projects need to know where to find the dependencies, thus making the RPMs relocatable requires much work. Is it possible though, to generate a separate set of RPMs for installation purposes, which are different than the ones generated currently. (Just an idea based on my limited understanding of BigTop).
Thanks. On Wed, Jul 9, 2014 at 3:12 PM, Andrew Purtell <[EMAIL PROTECTED]> wrote:
On Wed, Jul 9, 2014 at 3:12 PM, Andrew Purtell <[EMAIL PROTECTED]> wrote:
Truly relocatable DEBs are next to impossible. However, after having a chance to deal with this issue back at Cloudera, I'm now firmly convinced that somebody asking for relocatable packages is typically asking for two things: #1 be able to install different versions of the same package side-by-side #2 be able to install under a common subtree (such as /opt/our/hadoop)
In both of these cases, the package ends up being treated as a glorifies tarball. Why? Well, because: * pre/post install scriplets are downright *dangerous* in those scenarious * you have to do all the hooks to /etc/init.d &co manually anyway * you can't really use the goodness of yum repos & such.
If packages indeed are treated as a glorified tarballs -- what's wrong with dpkg -x pkg.deb /path and rpm2cpio pkg.rpm | cpio -i --make-directories ? Very much +1 to that! At Pivotal (being a home of a world renowned PaaS) we're looking into exactly that. A combination of Docker/LXC and OSv containerized deployments that you can 'bake' on the fly provide for some exciting opportunities. All of this, of course, comes at a price of breaking a traditional CM (Puppet, etc.) model of classical deployment.
Anyway, if there's a subset of folks who are interested in the next. gen deployment approaches (especially for ephemeral Hadoop clusters) I'd love to organize a meetup/pow-wow on that subject.
Hi Roman, I am currently doing some work in this area with app containers and would be very interested in getting-to-gether to brainstorm and to contribute in that area. Please let me know if there is going to be a meetup or a pow-wow on this subject. On Wed, Jul 9, 2014 at 10:30 PM, Roman Shaposhnik <[EMAIL PROTECTED]> wrote: