Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> as i understand rebalance happens on client side


Copy link to this message
-
Re: as i understand rebalance happens on client side
Currently we are preparing for the 0.8-final release. But 0.9 is not slated
to release until early next year.

Thanks,
Neha
On Tue, Oct 1, 2013 at 9:26 AM, Kane Kane <[EMAIL PROTECTED]> wrote:

> Btw, is it expected to be released on Oct 31?
>
> Thanks!
>
>
> On Tue, Oct 1, 2013 at 9:01 AM, Neha Narkhede <[EMAIL PROTECTED]
> >wrote:
>
> > We do plan to move the group membership over to the server side and have
> a
> > very thin consumer client. The proposal is here -
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Client+Rewrite#ClientRewrite-ConsumerAPIand
> > this is being planned for the 0.9 release. Once this is complete, the
> > non-java clients can easily leverage the group management feature.
> >
> > Thanks,
> > Neha
> >
> >
> > On Tue, Oct 1, 2013 at 8:56 AM, Kane Kane <[EMAIL PROTECTED]> wrote:
> >
> > > The reason i was asking is that this library seems to have support only
> > for
> > > SimpleConsumer https://github.com/mumrah/kafka-python/, i was curious
> if
> > > all should be implemented on client or kafka has some rebalancing logic
> > and
> > > prevent consuming from the same queue on server side in case of
> > > SimpleConsumer api, but I see now that everything should implemented on
> > the
> > > client side.
> > >
> > > Thanks.
> > >
> > >
> > > On Tue, Oct 1, 2013 at 8:52 AM, Guozhang Wang <[EMAIL PROTECTED]>
> > wrote:
> > >
> > > > I do not understand your question, what are you trying to implement?
> > > >
> > > >
> > > > On Tue, Oct 1, 2013 at 8:42 AM, Kane Kane <[EMAIL PROTECTED]>
> > wrote:
> > > >
> > > > > So essentially you can't do "queue" pattern, unless you somehow
> > > implement
> > > > > locking on the client?
> > > > >
> > > > >
> > > > > On Tue, Oct 1, 2013 at 8:35 AM, Guozhang Wang <[EMAIL PROTECTED]>
> > > > wrote:
> > > > >
> > > > > > SimpleConsumer do not have any concept of group management, only
> > the
> > > > > > high-level consumers have. So multiple simple consumers can
> > > > independently
> > > > > > consume from the same partition(s).
> > > > > >
> > > > > > Guozhang
> > > > > >
> > > > > >
> > > > > > On Tue, Oct 1, 2013 at 8:11 AM, Kane Kane <[EMAIL PROTECTED]
> >
> > > > wrote:
> > > > > >
> > > > > > > Yeah, I noticed that, i'm curious how balancing happens if
> > > > > SimpleConsumer
> > > > > > > is used. I.e. i can provide a partition to read from if i use
> > > > > > > SimpleConsumer, but what if someone else already attached to
> that
> > > > > > > partition, what would happen? Also what would happen if one
> > > > > > SimpleConsumer
> > > > > > > attached to all partitions? Noone would be able to join?
> > > > > > >
> > > > > > >
> > > > > > > On Tue, Oct 1, 2013 at 6:33 AM, Neha Narkhede <
> > > > [EMAIL PROTECTED]
> > > > > > > >wrote:
> > > > > > >
> > > > > > > > There are 2 types of consumer clients in Kafka -
> > > > > > > ZookeeperConsumerConnector
> > > > > > > > and SimpleConsumer. Only the former has the re balancing
> logic.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Neha
> > > > > > > > On Oct 1, 2013 6:30 AM, "Kane Kane" <[EMAIL PROTECTED]>
> > > wrote:
> > > > > > > >
> > > > > > > > > But it looks like some clients don't implement it?
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > -- Guozhang
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > -- Guozhang
> > > >
> > >
> >
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB