Kafka, mail # user - Streaming across data centers - 2013-06-04, 05:58
 Search Hadoop and all its subprojects:

Switch to Threaded View
Copy link to this message
-
Streaming across data centers
Hi guys,

I was wondering if someone here has an experience with streaming kafka
across data-centers?

How does kafka transfer the msgs? event by event or it batches some
events in one file(how many together?) and send them to the consumer on
the other data-center?

I am asking it, because we want to have a hadoop active-active clusters
(on two data-centers) and we might use the kafka as the sync stream from
which both centers will consume events,

Has anyone did something like that?

Thanks,
Guy Doulberg
Data Infra team leader @ Conduit
 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB