I am trying to deploy kafka cluster in two data centers, one being the
mirror of another. I have read that Linked-in also does similar kind of
mirroring. I am not sure about the zookeeper cluster deployment. Having
zookeeper clients talking over WAN is not recommended. Do you use zookeeper
observers?

Sharad

Sharad,

At LinkedIn, the mirroring cluster accesses ZK in another DC remotely. So
far, we haven't seen any problem with this setup. We haven't used ZK
observers yet, but it could be a good idea. One thing is that the mirroring
cluster also writes to the remote ZK to checkpoint the consumer offset. Not
sure if observers help here.

Jun

On Tue, Oct 11, 2011 at 3:04 AM, Sharad Agarwal <[EMAIL PROTECTED]>wrote:

> I am trying to deploy kafka cluster in two data centers, one being the
> mirror of another. I have read that Linked-in also does similar kind of
> mirroring. I am not sure about the zookeeper cluster deployment. Having
> zookeeper clients talking over WAN is not recommended. Do you use zookeeper
> observers?
>
> Sharad
>

In my understanding what is not recommended is having the zookeeper cluster
itself split across datacenters. The reason for this is that writes round
trip through all servers so that would add significantly to the write
latency. I haven't seen an equivalent recommendation on not accessing zk as
a client from another dc. As Jun says, it has worked well for us so far...

-Jay

On Tue, Oct 11, 2011 at 3:04 AM, Sharad Agarwal <[EMAIL PROTECTED]>wrote:

> I am trying to deploy kafka cluster in two data centers, one being the
> mirror of another. I have read that Linked-in also does similar kind of
> mirroring. I am not sure about the zookeeper cluster deployment. Having
> zookeeper clients talking over WAN is not recommended. Do you use zookeeper
> observers?
>
> Sharad
>

That's my understanding - zookeeper is not tolerant of the high (and more
important, variable, as opposed to the local dc) latencies in a WAN
environment.

On Tue, Oct 11, 2011 at 9:46 AM, Jay Kreps <[EMAIL PROTECTED]> wrote:

> In my understanding what is not recommended is having the zookeeper cluster
> itself split across datacenters. The reason for this is that writes round
> trip through all servers so that would add significantly to the write
> latency. I haven't seen an equivalent recommendation on not accessing zk as
> a client from another dc. As Jun says, it has worked well for us so far...
>
> -Jay
>
> On Tue, Oct 11, 2011 at 3:04 AM, Sharad Agarwal <[EMAIL PROTECTED]
> >wrote:
>
> > I am trying to deploy kafka cluster in two data centers, one being the
> > mirror of another. I have read that Linked-in also does similar kind of
> > mirroring. I am not sure about the zookeeper cluster deployment. Having
> > zookeeper clients talking over WAN is not recommended. Do you use
> zookeeper
> > observers?
> >
> > Sharad
> >
>

Thank you folks. I would admit the layout and examples are very nice. I
could bring things up on my laptop very quickly without reading much into
it.

I looked at the system test example as well. The ZK setup in source and
target clusters etc. I assume linked-in has the similar config. Each
DC/kafka cluster has its own ZK cluster. The mirroring consumer from another
DC points to the source ZK cluster.

Sharad
Hadoop Committer and Head of Technology Platforms@InMobi

Yes, what you describe is exactly our topology.

-Jay

On Tue, Oct 11, 2011 at 10:05 PM, Sharad Agarwal <[EMAIL PROTECTED]>wrote:

> Thank you folks. I would admit the layout and examples are very nice. I
> could bring things up on my laptop very quickly without reading much into
> it.
>
> I looked at the system test example as well. The ZK setup in source and
> target clusters etc. I assume linked-in has the similar config. Each
> DC/kafka cluster has its own ZK cluster. The mirroring consumer from
> another
> DC points to the source ZK cluster.
>
> Sharad
> Hadoop Committer and Head of Technology Platforms@InMobi
>