Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> changing broker hosts with 0.7.2


Copy link to this message
-
Re: changing broker hosts with 0.7.2
On Wed, Mar 20, 2013 at 10:55 AM, Jason Rosenberg <[EMAIL PROTECTED]> wrote:

> On Wed, Mar 20, 2013 at 9:06 AM, Philip O'Toole <[EMAIL PROTECTED]> wrote:
>
>> On Wed, Mar 20, 2013 at 8:41 AM, Jason Rosenberg <[EMAIL PROTECTED]> wrote:
>> > I think might be cool, would be to have a feature where by you can tell a
>> > broker to stop accepting new data produced to it, but still allow
>> consumers
>> > to consume from it.
>> >
>> > That way, you can roll out new brokers to a cluster, turn off producing
>> to
>> > the old nodes, then wait for the log retention period, and then remove
>> the
>> > old nodes from the cluster.
>> >
>> > Does that make sense?  Could it be easily done?
>>
>> Does to me. A way to do this is to place a load-balancer between your
>> consumers and brokers, allowing individual brokers to be taken out of
>> rotation for maintenance.
>
>
> Not sure I understand.  I need for a node to be taken out of the pool that
> producers produce to, but still need consumers consuming from all brokers,
> while we drain data from the brokers to be replaced.

Taking it out from behind the load-balancer between Producers and
Kafka means that Producers can no longer write to it. I said nothing
about disconnecting the *Consumers*. :-)

Philip

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB