Just curious - how are folks on this list deploying Kafka in a hosted production environment? Do you just unzip/build/run in place, on every broker machine? Or is there a more binary distribution approach that people are taking?

- Todd

On 06/11/2012 05:09 PM, Todd Bilsborrow wrote:
> Just curious - how are folks on this list deploying Kafka in a hosted production environment? Do you just unzip/build/run in place, on every broker machine? Or is there a more binary distribution approach that people are taking?
>
> - Todd
>

At AddThis it looks something like:
 - start with latest Apache release (or svn checkout)
 - Make a new custom tarball (remove contrib, custom jars, maybe change
defaults).
 - Have a separate set of templated config files (for brokers IDs etc.)
 - rsync everywhere
 - Pass off to whatever command and control tool [1] is cool these days.
[1] fabric, capistranto, shell scripts, whatever

BTW it would be great to get feedback on how we could improve this. The
linkedin folks are all using linkedin's internal deployment system, so we
haven't done much thinking in this area and personally I am not up on the
latest and greatest deployment tools.

-Jay

On Mon, Jun 18, 2012 at 6:41 PM, Chris Burroughs
<[EMAIL PROTECTED]>wrote:

> On 06/11/2012 05:09 PM, Todd Bilsborrow wrote:
> > Just curious - how are folks on this list deploying Kafka in a hosted
> production environment? Do you just unzip/build/run in place, on every
> broker machine? Or is there a more binary distribution approach that people
> are taking?
> >
> > - Todd
> >
>
> At AddThis it looks something like:
>  - start with latest Apache release (or svn checkout)
>  - Make a new custom tarball (remove contrib, custom jars, maybe change
> defaults).
>  - Have a separate set of templated config files (for brokers IDs etc.)
>  - rsync everywhere
>  - Pass off to whatever command and control tool [1] is cool these days.
>
>
> [1] fabric, capistranto, shell scripts, whatever
>

It might help to have a build script create a .zip file with all the
necessary files in it.

At Ooyala we are still using the binary distribution of 0.6, but we use a
Ruby deploy script to rsync the distribution + our own config files.

-Evan
On Mon, Jun 18, 2012 at 10:07 PM, Jay Kreps <[EMAIL PROTECTED]> wrote:

> BTW it would be great to get feedback on how we could improve this. The
> linkedin folks are all using linkedin's internal deployment system, so we
> haven't done much thinking in this area and personally I am not up on the
> latest and greatest deployment tools.
>
> -Jay
>
> On Mon, Jun 18, 2012 at 6:41 PM, Chris Burroughs
> <[EMAIL PROTECTED]>wrote:
>
> > On 06/11/2012 05:09 PM, Todd Bilsborrow wrote:
> > > Just curious - how are folks on this list deploying Kafka in a hosted
> > production environment? Do you just unzip/build/run in place, on every
> > broker machine? Or is there a more binary distribution approach that
> people
> > are taking?
> > >
> > > - Todd
> > >
> >
> > At AddThis it looks something like:
> >  - start with latest Apache release (or svn checkout)
> >  - Make a new custom tarball (remove contrib, custom jars, maybe change
> > defaults).
> >  - Have a separate set of templated config files (for brokers IDs etc.)
> >  - rsync everywhere
> >  - Pass off to whatever command and control tool [1] is cool these days.
> >
> >
> > [1] fabric, capistranto, shell scripts, whatever
> >
>

--
--
*Evan Chan*
Senior Software Engineer |
[EMAIL PROTECTED] | (650) 996-4600
www.ooyala.com | blog <http://www.ooyala.com/blog> |
@ooyala<http://www.twitter.com/ooyala>

we're creating a .rpm.  here's the spec-file if someone is interested:
https://gist.github.com/2052835

On Mon, Jun 11, 2012 at 11:09 PM, Todd Bilsborrow
<[EMAIL PROTECTED]> wrote:
> Just curious - how are folks on this list deploying Kafka in a hosted production environment? Do you just unzip/build/run in place, on every broker machine? Or is there a more binary distribution approach that people are taking?
>
> - Todd