Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka, mail # user - Changing the number of partitions after a topic is created


+
Calvin Lei 2013-07-04, 15:12
+
Jun Rao 2013-07-05, 04:06
+
Timothy Chen 2013-07-08, 17:33
+
Jun Rao 2013-07-09, 03:58
Copy link to this message
-
Re: Changing the number of partitions after a topic is created
Philip O'Toole 2013-07-04, 15:50
If you can pause your Producers, simply change the partition count to
10 in the Kafka server properties file, and restart it. If you use the
high-level consumer, it will automatically start draining all
partitions that exist for a given topic.

And our Kafka producers always write to partition -1 for a given
topic, meaning they never care about the partition count, and need no
changes in circumstances like this.

Philip

On Thu, Jul 4, 2013 at 11:11 AM, Calvin Lei <[EMAIL PROTECTED]> wrote:
> Hi I have a few topics created with 1 partition. After running the cluster
> for a few days, I want to increase the partition to 10 to improve the
> consumer throughput. I learnt that it is not supported in 0.8.
>
> What is the recommendation of improving consumer throughput after a topic
> is created and the data volume increased?
>
> Regards,
> Cal

 
+
Calvin Lei 2013-07-04, 16:06
+
Philip OToole 2013-07-04, 18:00
+
Jun Rao 2013-07-05, 19:08
+
Rajasekar Elango 2013-08-28, 15:47