Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> changing number of partitions


Copy link to this message
-
Re: changing number of partitions
> I mean, what happens to the partitions (or messages in them) that already
> exist above this number

We do not attempt to re-partition existing data within the partitions.
Discussion on this issue is here -
http://markmail.org/message/iyvtug6hlcykgbva?q=Repartitioning+list:org%2Eapache%2Eincubator%2Ekafka-users

> do the partitions 17-32 stay available to consumers, but new messages are
> just distributed across the first 16 from now on?

In 0.7, the extra partitions will stay available to consumers and the
data within those partitions will eventually be garbage collected by
the server, though
technically the server will never completely delete the directory
associated with the extra partitions.

In 0.8, we don't have the feature that allows you to change the number
of partitions yet, but we do plan to add it soon. JIRA is here -
https://issues.apache.org/jira/browse/KAFKA-347

Thanks,
Neha
On Thu, Nov 22, 2012 at 8:57 AM, Paul Garner
<[EMAIL PROTECTED]> wrote:
> what happens if I reduce num.partitions in server.properties?
>
> I mean, what happens to the partitions (or messages in them) that already
> exist above this number
>
> eg I have 32 partitions, messages have been distributed across them, then I
> set num.partitions=16
>
> do the partitions 17-32 stay available to consumers, but new messages are
> just distributed across the first 16 from now on?