Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Changing the number of partitions after a topic is created


Copy link to this message
-
Re: Changing the number of partitions after a topic is created
Thanks Phillip. I used the kafka-topic.sh to create the topic to have 1 partition. Would the changing the server properties still work in this case?

Also this setting sounds like a global overwrite, correct?

regards,
Cal
On Jul 4, 2013, at 11:49 AM, Philip O'Toole <[EMAIL PROTECTED]> wrote:

> If you can pause your Producers, simply change the partition count to
> 10 in the Kafka server properties file, and restart it. If you use the
> high-level consumer, it will automatically start draining all
> partitions that exist for a given topic.
>
> And our Kafka producers always write to partition -1 for a given
> topic, meaning they never care about the partition count, and need no
> changes in circumstances like this.
>
> Philip
>
> On Thu, Jul 4, 2013 at 11:11 AM, Calvin Lei <[EMAIL PROTECTED]> wrote:
>> Hi I have a few topics created with 1 partition. After running the cluster
>> for a few days, I want to increase the partition to 10 to improve the
>> consumer throughput. I learnt that it is not supported in 0.8.
>>
>> What is the recommendation of improving consumer throughput after a topic
>> is created and the data volume increased?
>>
>> Regards,
>> Cal
 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB