-Re: Number of consumers limited to number of partitions
Vaibhav Puranik 2012-10-11, 16:34
Thanks Neha and Jay.
It makes sense.
On Thu, Oct 11, 2012 at 8:41 AM, Jay Kreps <[EMAIL PROTECTED]> wrote:
> Yes, the reason for this is that if two consumers consumed the same
> partition internally we would have to keep track of per-message consumption
> (which is expensive) and the ordering guarantee would be broken (since the
> actual processing in the two consumer processes might happen out of order).
> Since you can always have more partitions this should not be a limitation,
> just a parameter to be aware of.
> On Thu, Oct 11, 2012 at 8:20 AM, Vaibhav Puranik <[EMAIL PROTECTED]>
> > Hi all,
> > We have a topic with three partitions - one on each broker (Kafka 0.7.1).
> > Using ConsumerConnector I noticed that I can consume messages from three
> > consumers at most. When I start fourth one, one of the older one stops.
> > Is this as designed? If I want more parallelism, do I need to increase
> > number of partitions?
> > Regards,
> > Vaibhav
> > GumGum