Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> partition reassignment


Copy link to this message
-
Re: partition reassignment
For a leader change yes, but this is partition reassignment which
completes when all the reassigned replicas are in sync with the
original replica(s). You can check the status of the command using the
option I mentioned earlier.

On Tue, Oct 15, 2013 at 7:02 PM, Kane Kane <[EMAIL PROTECTED]> wrote:
> I thought if i have all replicas in sync, leader change should be much
> faster?
>
>
> On Tue, Oct 15, 2013 at 5:12 PM, Joel Koshy <[EMAIL PROTECTED]> wrote:
>
>> Depending on how much data there is in those partitions it can take a
>> while for reassignment to actually complete. You will need to use the
>> --status-check-json-file option of the reassign partitions command to
>> determine whether partition reassignment has completed or not.
>>
>> Joel
>>
>>
>> On Tue, Oct 15, 2013 at 3:46 PM, Kane Kane <[EMAIL PROTECTED]> wrote:
>> > I have 3 brokers and a topic with replication factor of 3.
>> > Somehow all partitions ended up being on the same broker.
>> > I've created topic with 3 brokers alive, and they didn't die since then.
>> >
>> > Even when i try to reassign it:
>> > bin/kafka-reassign-partitions.sh --zookeeper
>> > 10.80.42.147:2181--broker-list 0,1,2 --topics-to-move-json-file
>> > ~/reassign.txt --execute
>> >
>> > The leader of all partitions after that is still the single broker (first
>> > one).
>> > This is how my json file looks like:
>> > {"topics":
>> >                                             [{"topic": "perf1"}],
>> >                                         "version":1
>> >                                         }
>> >
>> > The command reports success:
>> > Successfully started reassignment of partitions Map([perf1,33] -> List(1,
>> > 0, 2), [perf1,13] -> List(2, 0, 1), [perf1,28] -> List(2, 1, 0),
>> [perf1,20]
>> > -> List(0, 1, 2), [perf1,5] -> List(0, 2, 1), [perf1,14] -> List(0, 1,
>> 2),
>> > [perf1,39] -> List(1, 0, 2), [perf1,19] -> List(2, 0, 1), [perf1,25] ->
>> > List(2, 0, 1), [perf1,43] -> List(2, 0, 1), [perf1,11] -> List(0, 2, 1),
>> > [perf1,8] -> List(0, 1, 2), [perf1,22] -> List(2, 1, 0), [perf1,16] ->
>> > List(2, 1, 0), [perf1,26] -> List(0, 1, 2), [perf1,41] -> List(0, 2, 1),
>> > [perf1,9] -> List(1, 0, 2), [perf1,27] -> List(1, 0, 2), [perf1,44] ->
>> > List(0, 1, 2), [perf1,34] -> List(2, 1, 0), [perf1,6] -> List(1, 2, 0),
>> > [perf1,1] -> List(2, 0, 1), [perf1,37] -> List(2, 0, 1), [perf1,24] ->
>> > List(1, 2, 0), [perf1,29] -> List(0, 2, 1), [perf1,32] -> List(0, 1, 2),
>> > [perf1,7] -> List(2, 0, 1), [perf1,17] -> List(0, 2, 1), [perf1,36] ->
>> > List(1, 2, 0), [perf1,0] -> List(1, 2, 0), [perf1,42] -> List(1, 2, 0),
>> > [perf1,4] -> List(2, 1, 0), [perf1,10] -> List(2, 1, 0), [perf1,30] ->
>> > List(1, 2, 0), [perf1,23] -> List(0, 2, 1), [perf1,18] -> List(1, 2, 0),
>> > [perf1,3] -> List(1, 0, 2), [perf1,35] -> List(0, 2, 1), [perf1,31] ->
>> > List(2, 0, 1), [perf1,12] -> List(1, 2, 0), [perf1,2] -> List(0, 1, 2),
>> > [perf1,40] -> List(2, 1, 0), [perf1,38] -> List(0, 1, 2), [perf1,15] ->
>> > List(1, 0, 2), [perf1,21] -> List(1, 0, 2))
>> >
>> > But still nothing changes. I use ~/kafka-src/bin/kafka-topics.sh
>> > --zookeeper 10.80.42.147:2181 --describe --topic perf1 to check leaders.
>> >
>> > Thanks.
>>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB