Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Add a new node in an existing kafka 0.8 cluster


Copy link to this message
-
Re: Add a new node in an existing kafka 0.8 cluster
We fixed a few bugs in the reassignment logic and the 0.8 final release is
in progress. Recommend you use 0.8 final and see how that goes. We are
fixing it further in 0.8.1, so if you feel adventurous you can try using
trunk :)

Thanks
Neha
On Mon, Dec 2, 2013 at 2:12 PM, Maxime Nay <[EMAIL PROTECTED]> wrote:

> Ok, thanks.
> And now, let's say that I want to add two new nodes to increase the
> capacity of our cluster. Is that possible with the current beta1 release ?
>
>
> 2013/12/2 Neha Narkhede <[EMAIL PROTECTED]>
>
> > You can keep the broker.id of the new node same as the old node. Then it
> > will start up and copy everything from the leader for the partitions it
> is
> > assigned to. After it is caught up, you can run the
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Replication+tools#Replicationtools-2.PreferredReplicaLeaderElectionTooltool
> > to balance the leaders, so that the new node will have a few leaders
> > and will start serving data.
> >
> > Thanks,
> > Neha
> >
> >
> > On Mon, Dec 2, 2013 at 1:50 PM, Maxime Nay <[EMAIL PROTECTED]> wrote:
> >
> > > Hi,
> > >
> > > Last week we set-up a new kafka 0.8 cluster, using the beta1 released
> > > available here : http://kafka.apache.org/downloads.html
> > >
> > > It worked fine until we tried to replace a node in this cluster.
> > > We shutdown a node, then bring up a new one. The new node is registered
> > in
> > > zookeeper, but it doesn't get any data. (We have 10 nodes, 10
> partitions
> > > for each topic, and a replication factor of 3.)
> > >
> > > We saw that there are some tools available to balance the load, and add
> > > some broker to an existing cluster (
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Replication+tools#Replicationtools-ClusterExpansion
> > > ).
> > > But it seems they are not yet included in the Kafka version that we
> > > download from kafka.apache.org.
> > >
> > > Is there any branch where we can get a more current - production safe
> > > version where we can find the latest tools? Is it safe to get code from
> > > trunk and deploy it in production?
> > >
> > > Thanks
> > > Maxime
> > >
> >
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB