Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Streaming across data centers


Copy link to this message
-
Streaming across data centers
Hi guys,

I was wondering if someone here has an experience with streaming kafka
across data-centers?

How does kafka transfer the msgs? event by event or it batches some
events in one file(how many together?) and send them to the consumer on
the other data-center?

I am asking it, because we want to have a hadoop active-active clusters
(on two data-centers) and we might use the kafka as the sync stream from
which both centers will consume events,

Has anyone did something like that?

Thanks,
Guy Doulberg
Data Infra team leader @ Conduit
 
+
Jun Rao 2013-06-04, 14:48
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB