Kafka, mail # user - Re: Mirroring datacenters without vpn - 2014-01-11, 01:09
 Search Hadoop and all its subprojects:

Switch to Threaded View
Copy link to this message
-
Re: Mirroring datacenters without vpn

Would this feature help in your case:
https://issues.apache.org/jira/browse/KAFKA-1092
i.e., you can configure the broker to publish a separate hostname to
zookeeper which is what the producers should use when actually sending
data. So you would need to override the advertised.host.name and port
properties.
We generally run the mirror-maker in the target data center. i.e., we
do a remote consume but local produce. If you have a flaky connection
between the two clusters the consumers may encounter hit session
expirations and rebalance and reduce the overall throughput. You can
also do local consumption and remote produce although we have not
tried that. In either case you will need to set a high socket buffer
to help amortize the high network latencies.

Thanks,

Joel
 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB