-Re: Best practices for changing partition numbers
Corbin Hoenes 2013-01-08, 21:41
Sounds like there is an admin ddl you have to run on your existing topic to change the # after creation.
Sent from my iPhone
On Jan 8, 2013, at 2:24 PM, David Ross <[EMAIL PROTECTED]> wrote:
> Yeah that makes sense, but what if we do need to change the number of
> partitions? What if we need to reduce it?
> On Tue, Jan 8, 2013 at 12:42 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
>> If you don't have a lot of topics, one thing you can do is to
>> over-partition a topic.
>> Also, in 0.7, # of partitions grows with brokers. This is going to change
>> in 0.8, in which # of partitions is specified at topic creation time and
>> won't change as brokers change. One needs to use an admin DDL to change #
>> of partitions.
>> On Mon, Jan 7, 2013 at 10:23 PM, David Ross <[EMAIL PROTECTED]> wrote:
>>> We have found that, for our application, having a number of total
>>> partitions as a multiple of the number of consumer hosts is beneficial.
>>> Because of this, whenever we add or remove consumer hosts, we have to
>>> change the number of partitions in the server config.
>>> What are best practices for changing the number of partitions? It seems
>>> like adding partitions is fine but removing partitions would result in
>>> loss - am I right? Is that avoidable? Is it preferable to bring in new
>>> servers with new partitions? Anything else I should keep in mind on this