-Re: Consumer group and Partitions
Joel Koshy 2012-06-25, 17:47
That should be all you need - although you only need to bounce the kafka
broker. Are you sending enough messages? If you send only a few test
messages, it is possible for the messages to get sent to only one
partition. E.g., I tried a quick test with the ConsoleProducer and it took
three messages before the partition was created.
On Mon, Jun 25, 2012 at 8:43 AM, Narala, Ravi <[EMAIL PROTECTED]>wrote:
> Jun ,
> Thanks for the quick repsonse. I am running into a partition issue. I am
> not sure if am doing the right way. Here are the steps I did to create the
> partition. But I still do not see them partitioned.
> Stopped the kafka and zookeeper server
> Modified the num.partitions entry in server.properties from 1 to 2.
> Restarted the zookeeper and kafka server
> Used a Producer client to send messages to a topic named
> 'poc_test_topic1'.(I did not specify any partition key)
> The logs dir was configured to /tmp/kafka-logs in server.properties.
> I see only a 'poc_test_topic1-0' topic under logs dir. I was expecting to
> see 'poc_test_topic1-0' and 'poc_test_topic1-1' partitions under it.
> Am I missing something or is this all I need to create a partition ?
> Ravi Narala
> Web Services & Integration
> Level 3 Communications
> 200 Office Galleria Center
> Southfield, MI 48035 US
> p: 248-436-9324
> e: [EMAIL PROTECTED]
> -----Original Message-----
> From: Jun Rao [mailto:[EMAIL PROTECTED]]
> Sent: Monday, June 25, 2012 11:05 AM
> To: [EMAIL PROTECTED]
> Subject: Re: Consumer group and Partitions
> For 1, consumer group is specified in consumer config.
> For 2, partitions are divided evenly among consumers in the same group for
> consumption. For details, please see the design doc on our website.
> For 3, a producer never needs to know consumer groups. A producer always
> produces messages to a topic/partition. There are examples in the design
> doc too.
> On Mon, Jun 25, 2012 at 7:57 AM, Narala, Ravi <[EMAIL PROTECTED]
> > Hi,
> > I have couple of questions regarding the concepts of Consumer group
> > and partitions. Here are those ?
> > 1. When consuming a message how Do I know which consumer group I
> > belong to?. Is there a configuration available or do we have to
> > specify it at the application level ?
> > 2. How do consumer groups work in conjunction with partitions ?
> > 3. How do I produce a message to specific group and a partition ?.
> > coding example is greatly appreciated.
> > Thanks,
> > Ravi Narala
> > Web Services & Integration
> > Level 3 Communications
> > 200 Office Galleria Center
> > Southfield, MI 48035 US
> > p: 248-436-9324
> > e: [EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>