Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Kafka and Zookeeper node removal from two nodes Kafka cluster


Copy link to this message
-
Re: Kafka and Zookeeper node removal from two nodes Kafka cluster
Yeah, so it would seem a work around could be to defer full replica
assignment until adequate brokers are available, but in the meantime, allow
topic creation to proceed.

With respect to Joel's point around the possibility for imbalanced
partition assignment if not all replicas are available, this really just
points to the long-term need to have partition rebalancing (and I'd argue
this should be dynamic, and support heterogenous server nodes, where some
can have more capacity than others).  Ultimately, if we want to support
easy horizontal scalability, then it should be possible to quickly add new
nodes to a cluster, and have them in short-order start taking up equal
load, etc.
On Tue, Oct 15, 2013 at 8:57 PM, Jun Rao <[EMAIL PROTECTED]> wrote:

> When creating a new topic, we require # live brokers to be equal to or
> larger than # replicas. Without enough brokers, can't complete the replica
> assignment since we can't assign more than 1 replica on the same broker.
>
> Thanks,
>
> Jun
>
>
> On Tue, Oct 15, 2013 at 1:47 PM, Jason Rosenberg <[EMAIL PROTECTED]> wrote:
>
> > Is there a fundamental reason for not allowing creation of new topics
> while
> > in an under-replicated state?  For systems that use automatic topic
> > creation, it seems like losing a node in this case is akin to the cluster
> > being unavailable, if one of the nodes goes down, etc.
> >
> >
> > On Tue, Oct 15, 2013 at 1:25 PM, Joel Koshy <[EMAIL PROTECTED]> wrote:
> >
> > > Steve - that's right. I think Monika wanted clarification on what
> > > would happen if replication factor is two and only one broker is
> > > available. In that case, you won't be able to create new topics with
> > > replication factor two (you should see an AdministrationException
> > > saying the replication factor is larger than available brokers).
> > >
> > > However, you can send messages to available partitions of topics that
> > > have already been created - because the ISR would shrink to only one
> > > replica for those topics - although the cluster would be in an
> > > under-replicated state. This is covered in the documentation
> > > (http://kafka.apache.org/documentation.html#replication) under the
> > > discussion about ISR.
> > >
> > > Thanks,
> > >
> > > Joel
> > >
> > > On Tue, Oct 15, 2013 at 10:19 AM, Steve Morin <[EMAIL PROTECTED]>
> > > wrote:
> > > > If you have a double broker failure with replication factor of 2 and
> > only
> > > > have 2 brokers in the cluster.  Wouldn't every partition be not
> > > available?
> > > >
> > > >
> > > > On Tue, Oct 15, 2013 at 8:48 AM, Jun Rao <[EMAIL PROTECTED]> wrote:
> > > >
> > > >> If you have double broker failures with a replication factor of 2,
> > some
> > > >> partitions will not be available. When one of the brokers comes
> back,
> > > the
> > > >> partition is made available again (there is potential data loss),
> but
> > > in an
> > > >> under replicated mode. After the second broker comes back, it will
> > > catch up
> > > >> from the other replica and the partition will eventually be fully
> > > >> replicated. There is no need to change the replication factor during
> > > this
> > > >> process.
> > > >>
> > > >> As for ZK, you can always use the full connection string. ZK will
> pick
> > > live
> > > >> servers to establish connections.
> > > >>
> > > >> Thanks,
> > > >>
> > > >> Jun
> > > >>
> > > >>
> > > >> On Tue, Oct 15, 2013 at 3:46 AM, Monika Garg <[EMAIL PROTECTED]>
> > > wrote:
> > > >>
> > > >> > I have 2 nodes kafka cluster with default.replication.factor=2,is
> > set
> > > in
> > > >> > server.properties file.
> > > >> >
> > > >> > I removed one node-in removing that node,I killed Kafka
> > > process,removed
> > > >> all
> > > >> > the kafka-logs and bundle from that node.
> > > >> >
> > > >> > Then I stopped my remaining running node in the cluster and
> started
> > > >> > again(default.replication.factor is still set to 2 in this node
> > > >> > server.properties file).
> > > >> > I was expecting some error/exception as now I don't have two nodes

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB