Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka, mail # user - changing broker hosts with 0.7.2


+
Jason Rosenberg 2013-03-19, 21:04
+
Neha Narkhede 2013-03-20, 02:07
+
Jason Rosenberg 2013-03-20, 06:26
+
Neha Narkhede 2013-03-20, 13:42
+
Jason Rosenberg 2013-03-20, 15:56
+
Neha Narkhede 2013-03-20, 16:02
Copy link to this message
-
Re: changing broker hosts with 0.7.2
Philip O'Toole 2013-03-20, 16:07
On Wed, Mar 20, 2013 at 8:41 AM, Jason Rosenberg <[EMAIL PROTECTED]> wrote:
> I think might be cool, would be to have a feature where by you can tell a
> broker to stop accepting new data produced to it, but still allow consumers
> to consume from it.
>
> That way, you can roll out new brokers to a cluster, turn off producing to
> the old nodes, then wait for the log retention period, and then remove the
> old nodes from the cluster.
>
> Does that make sense?  Could it be easily done?

Does to me. A way to do this is to place a load-balancer between your
consumers and brokers, allowing individual brokers to be taken out of
rotation for maintenance.

>
> Or is all this a non-issue anyway in 0.8?
>
> Maybe I should just wait for 0.8 to be ready, before doing my migration
> anyway.....
>
> Jason
>
> On Wed, Mar 20, 2013 at 6:42 AM, Neha Narkhede <[EMAIL PROTECTED]>wrote:
>
>> The zookeeper connection URL with namespace can be
>> zkhost1:123,zkhost2:123/newnamespace
>>
>> The wiki is up to date for Kafka 0.7.2. There is no officially supported
>> feature to do that sort of migration, I suggested one approach that I could
>> think of :-)
>>
>> Thanks,
>> Neha
>>
>> On Tuesday, March 19, 2013, Jason Rosenberg wrote:
>>
>> > I can do most of that I presume.
>> >
>> > It looks like to set up a separate namespace for zk, I can add /path at
>> the
>> > end of each node:port in my zkconnect string, e.g.:
>> >  zkhost1:123/newnamespace,zkhost2:123/newnamespace
>> > right?
>> >
>> > For mirroring, there's some vague documentation here:
>> > https://cwiki.apache.org/KAFKA/kafka-mirroring-mirrormaker.html
>> > Is this the most up to date approach for 0.7.2?  Set up a MirrorMaker
>> > intermediate process that consumes from the old and produces to the new?
>> >
>> > I am not able to restart producers one by one (as there are many, on a
>> > rather asynchronous update/restart cycle).  But I can eventually get them
>> > migrated over, etc.
>> >
>> > Jason
>> >
>> > On Tue, Mar 19, 2013 at 7:07 PM, Neha Narkhede <[EMAIL PROTECTED]
>> <javascript:;>
>> > >wrote:
>> >
>> > > Can you do the following -
>> > >
>> > > 1. Start a mirror Kafka cluster with the new version on a separate
>> > > zookeeper namespace. Configure this to mirror data from the existing
>> > kafka
>> > > cluster.
>> > > 2. Move your consumers to pull data from the mirror
>> > > 3. For each producer, one at a time, change the zookeeper namespace to
>> > > point to the mirror and restart the producer.
>> > > 4. Once the producers have moved to mirror cluster, shutdown mirroring
>> > and
>> > > old cluster.
>> > >
>> > > Thanks,
>> > > Neha
>> > >
>> > > On Tuesday, March 19, 2013, Jason Rosenberg wrote:
>> > >
>> > > > I need to upgrade some kafka broker servers.  So I need to seamlessly
>> > > > migrate traffic from the old brokers to the new ones, without losing
>> > > data,
>> > > > and without stopping producers.  I can temporarily stop consumers,
>> etc.
>> > > >
>> > > > Is there a strategy for this?
>> > > >
>> > > > Also, because of the way we are embedding kafka in our framework, our
>> > > > brokerId's are auto-generated (based on hostname, etc.), so I can't
>> > > simply
>> > > > copy over broker log files, etc., by transferring an old brokerId to
>> a
>> > > new
>> > > > host.
>> > > >
>> > > > Is there a way to change the view of the cluster from the producer's
>> > > > standpoint, without doing so from the consumers standpoint?  That
>> way,
>> > > the
>> > > > producers can start writing to the new brokers, while the consumers
>> > drain
>> > > > all data from the old brokers before switching to the new brokers.
>> > > >
>> > > > I don't actually care about ordering of messages, since the consumers
>> > are
>> > > > publishing them to a store that will index them properly based on
>> > source
>> > > > timestamp, etc.
>> > > >
>> > > > We are using zk for both producers and consumers connections.
>> > > >
>> > > > This is using 0.7.2.  I assume in 0.8 it will be easier, since with

 
+
Jason Rosenberg 2013-03-20, 17:55
+
Philip OToole 2013-03-20, 18:10
+
Jason Rosenberg 2013-03-20, 18:21
+
Philip OToole 2013-03-20, 19:00
+
Jason Rosenberg 2013-03-20, 19:07
+
Philip OToole 2013-03-20, 19:16
+
Jason Rosenberg 2013-03-20, 19:33