Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> The future of 0.7


Copy link to this message
-
Re: The future of 0.7
Hi Jason,

This is included with Kafka 0.8 - kafka.tools.KafkaMigrationTool. It runs a
0.7 consumer and 0.8 producer to copy the data between your 0.7 Kafka
cluster and 0.8 Kafka cluster.

Thanks,
Neha
On Tue, Jan 22, 2013 at 12:21 PM, Jason Rosenberg <[EMAIL PROTECTED]> wrote:

> Hi Neha,
>
> Can you describe the migration tool you mention below, for copying data
> from 0.7 to 0.8?  Is this something provided with 0.8?  Or do apps need to
> write custom migration tools?
>
> Thanks,
>
> Jason
>
> On Tue, Jan 15, 2013 at 11:06 AM, Neha Narkhede <[EMAIL PROTECTED]
> >wrote:
>
> > Broadly, the strategy stays the same, except that the 0.8 broker will not
> > be able to serve requests coming from a pre-0.8 client.
> > So, the upgrade path will be and we will document this in detail soon -
> >
> > 1. Start a 0.8 Kafka cluster overlaid on the 0.6/0.7 hardware or on
> > separate hardware if you can spare
> > 2. Start a migration tool to copy data from 0.7 to 0.8. Currently we
> don't
> > have migration tool support for 0.6 but it is very easy to add, so please
> > file a JIRA for this.
> > 3. Move consumers to the 0.8 cluster (since it has all the data that your
> > 0.6/0.7 cluster has)
> > 4. Once all consumers have moved, start moving producers over to the 0.8
> > cluster
> > 5. Stop and decommission the 0.6/0.7 Kafka cluster.
> >
> > Thanks,
> > Neha
> >
> >
> > On Tue, Jan 15, 2013 at 11:01 AM, Evan Chan <[EMAIL PROTECTED]> wrote:
> >
> > > Will it be possible to upgrade from 0.6 directly to 0.8, and what would
> > the
> > > migration strategy be like?   I understand for 0.6 to 0.7, the strategy
> > was
> > > to first upgrade your consumers, then the broker, then the producers.
> > >
> > > -Evan
> > >
> > >
> > > On Tue, Jan 15, 2013 at 9:44 AM, Neha Narkhede <
> [EMAIL PROTECTED]
> > > >wrote:
> > >
> > > > To answer your question, even if the team is working on 0.8, we
> > > understand
> > > > that it will take some time for 0.8 to be completely stable and for
> > users
> > > > to build some confidence in moving to 0.8 since it is backwards
> > > > incompatible. Keeping that in mind, we will try our best to patch the
> > 0.7
> > > > branch for bugs and do point releases from there for a few months
> down
> > > the
> > > > line.
> > > > Feel free to file bugs and we can discuss how to handle a release for
> > > those
> > > > bug fixes.
> > > >
> > > > Thanks,
> > > > Neha
> > > >
> > > >
> > > > On Tue, Jan 15, 2013 at 9:39 AM, Jun Rao <[EMAIL PROTECTED]> wrote:
> > > >
> > > > > Our current plan to to get 0.8 out asap and only patch blocker
> issues
> > > in
> > > > > 0.7. If the issues that you found apply to 0.8, please feel free to
> > > open
> > > > a
> > > > > jira and/or attach a patch.
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Jun
> > > > >
> > > > > On Tue, Jan 15, 2013 at 9:07 AM, Wouter de Bie <[EMAIL PROTECTED]
> >
> > > > wrote:
> > > > >
> > > > > > Hi all,
> > > > > >
> > > > > > We're currently using Kafka 0.7 and here and there we're running
> > in a
> > > > few
> > > > > > bugs here and there. Since 0.8 is around the corner, it might not
> > > make
> > > > > too
> > > > > > much sense in sending the patches for 0.8 upstream, unless 0.7
> will
> > > > live
> > > > > a
> > > > > > (long) life next to 0.8. What's your take on the future of the
> 0.7
> > > > > branch?
> > > > > >
> > > > > > Thanks!
> > > > > >
> > > > > > --
> > > > > > Wouter de Bie
> > > > > > Team Lead Analytics and Data Infrastructure, Spotify
> > > > > > [EMAIL PROTECTED]
> > > > > > +46 72 018 0777
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > --
> > > *Evan Chan*
> > > Senior Software Engineer |
> > > [EMAIL PROTECTED] | (650) 996-4600
> > > www.ooyala.com | blog <http://www.ooyala.com/blog> |
> > > @ooyala<http://www.twitter.com/ooyala>
> > >
> >
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB