-Re: Kafka mirroring fault tolerance
Jun Rao 2013-03-23, 03:58
Yes, this is true if you have only 1 broker in the target cluster. If you
set up multiple brokers in the target cluster, mirror maker will send
messages to available brokers.
On Fri, Mar 22, 2013 at 12:02 PM, Riju Kallivalappil <
[EMAIL PROTECTED]> wrote:
> I've a question about fault tolerance of Kafka mirror maker (0.7.1)
> Let's say that I've a mirroring setup with topics in broker B1 mirrored to
> broker B2.
> On B2, I've Kafka mirror maker and Kafka broker process running. Now,
> following is what I noticed when the Kafka broker process on B2 is
> The embedded consumer in the Mirror Maker continues to consume messages
> from B1 and updates the ZK offsets even after the Kafka process is stopped.
> Pretty much all of the messages that are consumed during the time that the
> local broker process is down is lost.
> Is this expected behavior? Looking through the code, I suppose it is.
> Are there any suggestions on how to avoid this kind of data loss?