Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> partition reassignment


Copy link to this message
-
Re: partition reassignment
Depending on how much data there is in those partitions it can take a
while for reassignment to actually complete. You will need to use the
--status-check-json-file option of the reassign partitions command to
determine whether partition reassignment has completed or not.

Joel
On Tue, Oct 15, 2013 at 3:46 PM, Kane Kane <[EMAIL PROTECTED]> wrote:
> I have 3 brokers and a topic with replication factor of 3.
> Somehow all partitions ended up being on the same broker.
> I've created topic with 3 brokers alive, and they didn't die since then.
>
> Even when i try to reassign it:
> bin/kafka-reassign-partitions.sh --zookeeper
> 10.80.42.147:2181--broker-list 0,1,2 --topics-to-move-json-file
> ~/reassign.txt --execute
>
> The leader of all partitions after that is still the single broker (first
> one).
> This is how my json file looks like:
> {"topics":
>                                             [{"topic": "perf1"}],
>                                         "version":1
>                                         }
>
> The command reports success:
> Successfully started reassignment of partitions Map([perf1,33] -> List(1,
> 0, 2), [perf1,13] -> List(2, 0, 1), [perf1,28] -> List(2, 1, 0), [perf1,20]
> -> List(0, 1, 2), [perf1,5] -> List(0, 2, 1), [perf1,14] -> List(0, 1, 2),
> [perf1,39] -> List(1, 0, 2), [perf1,19] -> List(2, 0, 1), [perf1,25] ->
> List(2, 0, 1), [perf1,43] -> List(2, 0, 1), [perf1,11] -> List(0, 2, 1),
> [perf1,8] -> List(0, 1, 2), [perf1,22] -> List(2, 1, 0), [perf1,16] ->
> List(2, 1, 0), [perf1,26] -> List(0, 1, 2), [perf1,41] -> List(0, 2, 1),
> [perf1,9] -> List(1, 0, 2), [perf1,27] -> List(1, 0, 2), [perf1,44] ->
> List(0, 1, 2), [perf1,34] -> List(2, 1, 0), [perf1,6] -> List(1, 2, 0),
> [perf1,1] -> List(2, 0, 1), [perf1,37] -> List(2, 0, 1), [perf1,24] ->
> List(1, 2, 0), [perf1,29] -> List(0, 2, 1), [perf1,32] -> List(0, 1, 2),
> [perf1,7] -> List(2, 0, 1), [perf1,17] -> List(0, 2, 1), [perf1,36] ->
> List(1, 2, 0), [perf1,0] -> List(1, 2, 0), [perf1,42] -> List(1, 2, 0),
> [perf1,4] -> List(2, 1, 0), [perf1,10] -> List(2, 1, 0), [perf1,30] ->
> List(1, 2, 0), [perf1,23] -> List(0, 2, 1), [perf1,18] -> List(1, 2, 0),
> [perf1,3] -> List(1, 0, 2), [perf1,35] -> List(0, 2, 1), [perf1,31] ->
> List(2, 0, 1), [perf1,12] -> List(1, 2, 0), [perf1,2] -> List(0, 1, 2),
> [perf1,40] -> List(2, 1, 0), [perf1,38] -> List(0, 1, 2), [perf1,15] ->
> List(1, 0, 2), [perf1,21] -> List(1, 0, 2))
>
> But still nothing changes. I use ~/kafka-src/bin/kafka-topics.sh
> --zookeeper 10.80.42.147:2181 --describe --topic perf1 to check leaders.
>
> Thanks.

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB