Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Partition reassignment stucked


Copy link to this message
-
Re: Partition reassignment stucked
We will announce it on this mailing list. It is probably a month away from
a release.

Thanks,
Neha
On Mon, Dec 9, 2013 at 12:02 PM, Maxime Nay <[EMAIL PROTECTED]> wrote:

> Ok, thanks for your help. When 0.8.1 will be production ready, will you
> announce it somewhere (will you release it right away) ?
>
> Thanks,
> Maxime
>
>
>
> 2013/12/9 Neha Narkhede <[EMAIL PROTECTED]>
>
> > I wouldn't call 0.8.1 production ready just yet. We are still in the
> > process of deploying it at LinkedIn. Until it is ready, there isn't a
> good
> > cluster expansion solution other than spinning up a new cluster. This is
> > probably a little easier if you have a VIP in front of your kafka
> cluster.
> >
> > Thanks,
> > Neha
> >
> >
> > On Mon, Dec 9, 2013 at 9:40 AM, Maxime Nay <[EMAIL PROTECTED]> wrote:
> >
> > > Hi,
> > >
> > > We used the code checked in this branch a few hours before the official
> > > 0.8.0 final release : https://github.com/apache/kafka/tree/0.8
> > > So hopefully it should be the exact same code as the official release.
> > >
> > > The controller logs are empty.
> > >
> > > In a previous exchange you advised us to not use trunk unless we felt
> > > adventurous.
> > > So if you think 0.8.1 is production ready, and that it shouldn't crash
> > > unexpectedly, we can probably give it a try. Otherwise, since our data
> > > pipeline heavily relies on Kafka, we might have to use another
> > solution...
> > > (bring up a brand new cluster each time we want to resize the cluster
> ?)
> > >
> > > Thanks
> > > Maxime
> > >
> > >
> > >
> > > 2013/12/8 Neha Narkhede <[EMAIL PROTECTED]>
> > >
> > > > Unfortunately quite a few bugs with reassignment are fixed only in
> > > 0.8.1. I
> > > > wonder if you can run trunk and see how that goes?
> > > >
> > > > Thanks,
> > > > Neha
> > > > On Dec 6, 2013 9:46 PM, "Jun Rao" <[EMAIL PROTECTED]> wrote:
> > > >
> > > > > Are you using the 0.8.0 final release? Any error in controller log?
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Jun
> > > > >
> > > > >
> > > > > On Fri, Dec 6, 2013 at 4:38 PM, Maxime Nay <[EMAIL PROTECTED]>
> > > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > We are trying to add a broker to a 10 node cluster. We have 7
> > > different
> > > > > > topics, each of them is divided in 10 partitions, and their
> > > replication
> > > > > > factor is 3.
> > > > > >
> > > > > > To send traffic to this new node, we tried the
> > > > > kafka-reassign-partitions.sh
> > > > > > tool, but for some reason, it doesnt work, and now it seems that
> we
> > > are
> > > > > > stuck in some kind of partition reassignment process.
> > > > > > We can't see any data on our new node, and when we try to execute
> > the
> > > > > > partition reassignment tool for another topic it says :
> > > > > >
> > > > > > Partitions reassignment failed due to Partition reassignment
> > > currently
> > > > in
> > > > > > progress for Map([AD_EVENTS,5] ->
> > > > > > ReassignedPartitionsContext(List(2),null), [AD_EVENTS,3] ->
> > > > > > ReassignedPartitionsContext(List(1),null), [AD_EVENTS,4] ->
> > > > > > ReassignedPartitionsContext(List(3),null)). Aborting operation
> > > > > > kafka.common.AdminCommandFailedException: Partition reassignment
> > > > > currently
> > > > > > in progress for Map([AD_EVENTS,5] ->
> > > > > > ReassignedPartitionsContext(List(2),null), [AD_EVENTS,3] ->
> > > > > > ReassignedPartitionsContext(List(1),null), [AD_EVENTS,4] ->
> > > > > > ReassignedPartitionsContext(List(3),null)). Aborting operation
> > > > > >     at
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> kafka.admin.ReassignPartitionsCommand.reassignPartitions(ReassignPartitionsCommand.scala:201)
> > > > > >     at
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> kafka.admin.ReassignPartitionsCommand$.main(ReassignPartitionsCommand.scala:140)
> > > > > >     at
> > > > > >
> > > > >
> > > >
> > >
> >
> kafka.admin.ReassignPartitionsCommand.main(ReassignPartitionsCommand.scala)
> >
 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB