Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Re: Producer not distributing across all partitions


Copy link to this message
-
Re: Producer not distributing across all partitions
Which revision of 0.8 are you using? In a recent change, a producer will
stick to a partition for topic.metadata.refresh.interval.ms (defaults to 10
mins) time before picking another partition at random.
Thanks,
Jun
On Thu, Sep 12, 2013 at 1:56 PM, prashant amar <[EMAIL PROTECTED]> wrote:

> I created a topic with 4 partitions and for some reason the producer is
> pushing only to one partition.
>
> This is consistently happening across all topics that I created ...
>
> Is there a specific configuration that I need to apply to ensure that load
> is evenly distributed across all partitions?
>
>
> Group           Topic                          Pid Offset          logSize
>         Lag             Owner
> perfgroup1      perfpayload1                   0   10965           11220
>         255             perfgroup1_XXXX-0
> perfgroup1      perfpayload1                   1   0               0
>         0               perfgroup1_XXXX-1
> perfgroup1      perfpayload1                   2   0               0
>         0               perfgroup1_XXXXX-2
> perfgroup1      perfpayload1                   3   0               0
>         0               perfgroup1_XXXXX-3
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB