Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Does Kafka stores Producer/Consumer Details


+
Monika Garg 2013-10-25, 06:14
Copy link to this message
-
Re: Does Kafka stores Producer/Consumer Details
>
> I am wondering does metadata of Kafka stores producer/consumer information.
>
> As I start any producer,I provide a broker-list to it.But producer must be
> connecting to one of the broker to send data(as per Im gettiing the
> things).Similarly I can start more producer by giving a broker-list in the
> start command.
>
> So a broker must be storing the information about which producer is
> handshaking with it(as a producer will connect to only one broker at any
> time).

Not really - well, brokers obviously have to maintain these
connections from producers but it maintains no additional per-producer
state.

>
> Same case goes with consumer.

Same for consumers. For high-level consumers, the consumers themselves
maintain state (e.g., registration, partition ownership, offsets) in
zookeeper but there is no per-consumer state on the broker (although
with KAFKA-1000, we will be moving offset management to the broker).

Thanks,

Joel

 
+
Jun Rao 2013-10-25, 14:56
+
Monika Garg 2013-10-25, 15:34
+
Joel Koshy 2013-10-26, 01:01
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB