Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - Question about auto-rebalancing


Copy link to this message
-
Re: Question about auto-rebalancing
Jun Rao 2013-10-11, 15:24
If you are using simple consumer, you are responsible for dealing with
leader replica changes. When the leader changes, an error code will be
returned in the fetch response and you need to refresh metadata and retry
the fetch request to the new leader. For details, see
https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+SimpleConsumer+Example

Thanks,

Jun
On Thu, Oct 10, 2013 at 11:08 PM, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote:

> Hi guys,
>
> Here is a case I observed, I have a single-node 3 broker instance cluster.
> I created 1 topic with 2 partitions and 2 replica for each partition. The
> initial distribution is like this topic1/partition0 ->(broker0, broker2)
>  topic1/partition1 ->(broker1,broker2). So broker0 is leader broker for
> partition0 and broker1 is the leader broker for partition1.  I then kill
> broker0, broker3 becomes leader broker for partition1, then I kill broker2,
> broker1 becomes leader broker of both partition0 and partition1 which is
> fine.  But when I restart broker0, broker2, after they synced with broker1,
> they are just replica broker for partition0 and partition1. So my
> consumers(simple consumer) really don't know which broker it should read
> from. I found a command to that will force re-balance after failover, but
> isn't there any automatic way to rebalance the lleader broker?
>
> Best regards,
> Siyuan
>