Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - Re: Producer not distributing across all partitions


Copy link to this message
-
Re: Producer not distributing across all partitions
Jun Rao 2013-09-13, 09:11
Which revision of 0.8 are you using? In a recent change, a producer will
stick to a partition for topic.metadata.refresh.interval.ms (defaults to 10
mins) time before picking another partition at random.
Thanks,
Jun
On Thu, Sep 12, 2013 at 1:56 PM, prashant amar <[EMAIL PROTECTED]> wrote:

> I created a topic with 4 partitions and for some reason the producer is
> pushing only to one partition.
>
> This is consistently happening across all topics that I created ...
>
> Is there a specific configuration that I need to apply to ensure that load
> is evenly distributed across all partitions?
>
>
> Group           Topic                          Pid Offset          logSize
>         Lag             Owner
> perfgroup1      perfpayload1                   0   10965           11220
>         255             perfgroup1_XXXX-0
> perfgroup1      perfpayload1                   1   0               0
>         0               perfgroup1_XXXX-1
> perfgroup1      perfpayload1                   2   0               0
>         0               perfgroup1_XXXXX-2
> perfgroup1      perfpayload1                   3   0               0
>         0               perfgroup1_XXXXX-3
>