Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - partition reassignment


Copy link to this message
-
Re: partition reassignment
Joel Koshy 2013-10-16, 00:13
Depending on how much data there is in those partitions it can take a
while for reassignment to actually complete. You will need to use the
--status-check-json-file option of the reassign partitions command to
determine whether partition reassignment has completed or not.

Joel
On Tue, Oct 15, 2013 at 3:46 PM, Kane Kane <[EMAIL PROTECTED]> wrote:
> I have 3 brokers and a topic with replication factor of 3.
> Somehow all partitions ended up being on the same broker.
> I've created topic with 3 brokers alive, and they didn't die since then.
>
> Even when i try to reassign it:
> bin/kafka-reassign-partitions.sh --zookeeper
> 10.80.42.147:2181--broker-list 0,1,2 --topics-to-move-json-file
> ~/reassign.txt --execute
>
> The leader of all partitions after that is still the single broker (first
> one).
> This is how my json file looks like:
> {"topics":
>                                             [{"topic": "perf1"}],
>                                         "version":1
>                                         }
>
> The command reports success:
> Successfully started reassignment of partitions Map([perf1,33] -> List(1,
> 0, 2), [perf1,13] -> List(2, 0, 1), [perf1,28] -> List(2, 1, 0), [perf1,20]
> -> List(0, 1, 2), [perf1,5] -> List(0, 2, 1), [perf1,14] -> List(0, 1, 2),
> [perf1,39] -> List(1, 0, 2), [perf1,19] -> List(2, 0, 1), [perf1,25] ->
> List(2, 0, 1), [perf1,43] -> List(2, 0, 1), [perf1,11] -> List(0, 2, 1),
> [perf1,8] -> List(0, 1, 2), [perf1,22] -> List(2, 1, 0), [perf1,16] ->
> List(2, 1, 0), [perf1,26] -> List(0, 1, 2), [perf1,41] -> List(0, 2, 1),
> [perf1,9] -> List(1, 0, 2), [perf1,27] -> List(1, 0, 2), [perf1,44] ->
> List(0, 1, 2), [perf1,34] -> List(2, 1, 0), [perf1,6] -> List(1, 2, 0),
> [perf1,1] -> List(2, 0, 1), [perf1,37] -> List(2, 0, 1), [perf1,24] ->
> List(1, 2, 0), [perf1,29] -> List(0, 2, 1), [perf1,32] -> List(0, 1, 2),
> [perf1,7] -> List(2, 0, 1), [perf1,17] -> List(0, 2, 1), [perf1,36] ->
> List(1, 2, 0), [perf1,0] -> List(1, 2, 0), [perf1,42] -> List(1, 2, 0),
> [perf1,4] -> List(2, 1, 0), [perf1,10] -> List(2, 1, 0), [perf1,30] ->
> List(1, 2, 0), [perf1,23] -> List(0, 2, 1), [perf1,18] -> List(1, 2, 0),
> [perf1,3] -> List(1, 0, 2), [perf1,35] -> List(0, 2, 1), [perf1,31] ->
> List(2, 0, 1), [perf1,12] -> List(1, 2, 0), [perf1,2] -> List(0, 1, 2),
> [perf1,40] -> List(2, 1, 0), [perf1,38] -> List(0, 1, 2), [perf1,15] ->
> List(1, 0, 2), [perf1,21] -> List(1, 0, 2))
>
> But still nothing changes. I use ~/kafka-src/bin/kafka-topics.sh
> --zookeeper 10.80.42.147:2181 --describe --topic perf1 to check leaders.
>
> Thanks.