Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> The future of 0.7


+
Wouter de Bie 2013-01-15, 17:08
+
Jun Rao 2013-01-15, 17:40
+
Neha Narkhede 2013-01-15, 17:44
+
Evan Chan 2013-01-15, 19:02
+
Neha Narkhede 2013-01-15, 19:06
+
Jason Rosenberg 2013-01-22, 20:21
Copy link to this message
-
Re: The future of 0.7
Hi Jason,

This is included with Kafka 0.8 - kafka.tools.KafkaMigrationTool. It runs a
0.7 consumer and 0.8 producer to copy the data between your 0.7 Kafka
cluster and 0.8 Kafka cluster.

Thanks,
Neha
On Tue, Jan 22, 2013 at 12:21 PM, Jason Rosenberg <[EMAIL PROTECTED]> wrote:

> Hi Neha,
>
> Can you describe the migration tool you mention below, for copying data
> from 0.7 to 0.8?  Is this something provided with 0.8?  Or do apps need to
> write custom migration tools?
>
> Thanks,
>
> Jason
>
> On Tue, Jan 15, 2013 at 11:06 AM, Neha Narkhede <[EMAIL PROTECTED]
> >wrote:
>
> > Broadly, the strategy stays the same, except that the 0.8 broker will not
> > be able to serve requests coming from a pre-0.8 client.
> > So, the upgrade path will be and we will document this in detail soon -
> >
> > 1. Start a 0.8 Kafka cluster overlaid on the 0.6/0.7 hardware or on
> > separate hardware if you can spare
> > 2. Start a migration tool to copy data from 0.7 to 0.8. Currently we
> don't
> > have migration tool support for 0.6 but it is very easy to add, so please
> > file a JIRA for this.
> > 3. Move consumers to the 0.8 cluster (since it has all the data that your
> > 0.6/0.7 cluster has)
> > 4. Once all consumers have moved, start moving producers over to the 0.8
> > cluster
> > 5. Stop and decommission the 0.6/0.7 Kafka cluster.
> >
> > Thanks,
> > Neha
> >
> >
> > On Tue, Jan 15, 2013 at 11:01 AM, Evan Chan <[EMAIL PROTECTED]> wrote:
> >
> > > Will it be possible to upgrade from 0.6 directly to 0.8, and what would
> > the
> > > migration strategy be like?   I understand for 0.6 to 0.7, the strategy
> > was
> > > to first upgrade your consumers, then the broker, then the producers.
> > >
> > > -Evan
> > >
> > >
> > > On Tue, Jan 15, 2013 at 9:44 AM, Neha Narkhede <
> [EMAIL PROTECTED]
> > > >wrote:
> > >
> > > > To answer your question, even if the team is working on 0.8, we
> > > understand
> > > > that it will take some time for 0.8 to be completely stable and for
> > users
> > > > to build some confidence in moving to 0.8 since it is backwards
> > > > incompatible. Keeping that in mind, we will try our best to patch the
> > 0.7
> > > > branch for bugs and do point releases from there for a few months
> down
> > > the
> > > > line.
> > > > Feel free to file bugs and we can discuss how to handle a release for
> > > those
> > > > bug fixes.
> > > >
> > > > Thanks,
> > > > Neha
> > > >
> > > >
> > > > On Tue, Jan 15, 2013 at 9:39 AM, Jun Rao <[EMAIL PROTECTED]> wrote:
> > > >
> > > > > Our current plan to to get 0.8 out asap and only patch blocker
> issues
> > > in
> > > > > 0.7. If the issues that you found apply to 0.8, please feel free to
> > > open
> > > > a
> > > > > jira and/or attach a patch.
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Jun
> > > > >
> > > > > On Tue, Jan 15, 2013 at 9:07 AM, Wouter de Bie <[EMAIL PROTECTED]
> >
> > > > wrote:
> > > > >
> > > > > > Hi all,
> > > > > >
> > > > > > We're currently using Kafka 0.7 and here and there we're running
> > in a
> > > > few
> > > > > > bugs here and there. Since 0.8 is around the corner, it might not
> > > make
> > > > > too
> > > > > > much sense in sending the patches for 0.8 upstream, unless 0.7
> will
> > > > live
> > > > > a
> > > > > > (long) life next to 0.8. What's your take on the future of the
> 0.7
> > > > > branch?
> > > > > >
> > > > > > Thanks!
> > > > > >
> > > > > > --
> > > > > > Wouter de Bie
> > > > > > Team Lead Analytics and Data Infrastructure, Spotify
> > > > > > [EMAIL PROTECTED]
> > > > > > +46 72 018 0777
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > --
> > > *Evan Chan*
> > > Senior Software Engineer |
> > > [EMAIL PROTECTED] | (650) 996-4600
> > > www.ooyala.com | blog <http://www.ooyala.com/blog> |
> > > @ooyala<http://www.twitter.com/ooyala>
> > >
> >
>