Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Streaming across data centers


Copy link to this message
-
Re: Streaming across data centers
You can read the following wiki on the MirrorMaker tool.

https://cwiki.apache.org/confluence/display/KAFKA/Kafka+mirroring+%28MirrorMaker%29

Thanks,

Jun
On Mon, Jun 3, 2013 at 10:58 PM, Guy Doulberg <[EMAIL PROTECTED]>wrote:

> Hi guys,
>
> I was wondering if someone here has an experience with streaming kafka
> across data-centers?
>
> How does kafka transfer the msgs? event by event or it batches some events
> in one file(how many together?) and send them to the consumer on the other
> data-center?
>
> I am asking it, because we want to have a hadoop active-active clusters
> (on two data-centers) and we might use the kafka as the sync stream from
> which both centers will consume events,
>
> Has anyone did something like that?
>
> Thanks,
> Guy Doulberg
> Data Infra team leader @ Conduit
>
>
>
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB