Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Changing the number of partitions after a topic is created


+
Calvin Lei 2013-07-04, 15:12
+
Jun Rao 2013-07-05, 04:06
+
Timothy Chen 2013-07-08, 17:33
+
Jun Rao 2013-07-09, 03:58
Copy link to this message
-
Re: Changing the number of partitions after a topic is created
If you can pause your Producers, simply change the partition count to
10 in the Kafka server properties file, and restart it. If you use the
high-level consumer, it will automatically start draining all
partitions that exist for a given topic.

And our Kafka producers always write to partition -1 for a given
topic, meaning they never care about the partition count, and need no
changes in circumstances like this.

Philip

On Thu, Jul 4, 2013 at 11:11 AM, Calvin Lei <[EMAIL PROTECTED]> wrote:
> Hi I have a few topics created with 1 partition. After running the cluster
> for a few days, I want to increase the partition to 10 to improve the
> consumer throughput. I learnt that it is not supported in 0.8.
>
> What is the recommendation of improving consumer throughput after a topic
> is created and the data volume increased?
>
> Regards,
> Cal

 
+
Calvin Lei 2013-07-04, 16:06
+
Philip OToole 2013-07-04, 18:00
+
Jun Rao 2013-07-05, 19:08
+
Rajasekar Elango 2013-08-28, 15:47
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB