Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - Partitions and highlevel consumers


Copy link to this message
-
Partitions and highlevel consumers
Vinicius Carvalho 2013-07-02, 18:55
Hi guys, we are starting with kafka in our project. We are using version
0.8. I come from a traditional JMS MoM architecture, and some things are
new to me.

One thing that I'm not getting is the mapping between partitions and number
of threads. On a single consumer I can see the relationship but what
happens when you have multiple consumers?

So say, I have a topic and I have 5 instances of services that acts as
consumers. On a traditional pub/sub I would have all 5 instances to consume
the message right? Even if this topic only has one partition and now we
have 5 threads (one per jvm instance) consuming it. Does this proceed?

Just want to make sure if that is the expected behavior, from the docs it
wasn't clear if its the number of threads per consumer, or the total number
of threads around all clients to the topic.

Regards

--
The intuitive mind is a sacred gift and the
rational mind is a faithful servant. We have
created a society that honors the servant and
has forgotten the gift.