Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> consuming only half the messages produced


Copy link to this message
-
consuming only half the messages produced
Running a consumer group (createStreams()), pointing to the zookeeper and
with the topic and 1 consumer thread, results in only half the messages
being consumed.  The topic was auto-created, with a replication factor of 2,
but the producer was configured to produce to 2 brokers and so 4 partitions
resulted.  Are half getting sent to one leader, in one broker, and the other
half getting sent to another leader, in the other broker, but the consumer
stream is only reading from one leader from the zk?  Shouldn't there only be
one leader?

 

thanks,

rob