Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Re: kafka replication blog


Copy link to this message
-
Re: kafka replication blog
Thanks, makes sense.
On Feb 8, 2013 4:00 PM, "Jun Rao" <[EMAIL PROTECTED]> wrote:

> That's right. If you are partitioning by key, that means you insist a
> message has to go to a certain partition, whether it's available or not.
> So, if a partition is not available, we will drop the message for the
> partition in the async mode and consistently throw an exception to the
> caller in the sync mode.
>
> Thanks,
>
> Jun
>
> On Fri, Feb 8, 2013 at 1:41 AM, Michal Haris <[EMAIL PROTECTED]
> >wrote:
>
> > So if the produces are partitioning by key we have to have replication if
> > we dont want messages to get lost when partition goes down l right ?
> > Thanks
> > On Feb 8, 2013 5:12 AM, "Jun Rao" <[EMAIL PROTECTED]> wrote:
> >
> > > We have fixed this issue in 0.8. Withreplication factor 1, if the
> > producer
> > > doesn't care about partitioning by key, messages will be sent to
> > partitions
> > > that are currently available.
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > > On Thu, Feb 7, 2013 at 3:11 PM, Michal Haris <
> [EMAIL PROTECTED]
> > > >wrote:
> > >
> > > > Same here, summary was need as we have a fairly large ecosystem of
> > > multiple
> > > > 0.7.2 clusters and I am planning to test upgrade to 0.8.
> > > > However, one thing  creeping at the back of my mind regarding 0.8 is
> > > > something i have spotted in one thread few weeks ago namely that the
> > > > rebalance behaviour of producers is not as robust as in 0.7.x without
> > > > replication and i remeber there was no designed solution at the time
> -
> > > any
> > > > news here ? Basically our usecase doesn't require replication but
> > logical
> > > > offsets and some other things introduced would solve some problems.
> > > > On Feb 7, 2013 7:11 PM, "Vaibhav Puranik" <[EMAIL PROTECTED]>
> wrote:
> > > >
> > > > > Same here. Thanks a lot Jun.
> > > > >
> > > > > Regards,
> > > > > Vaibhav
> > > > >
> > > > > On Thu, Feb 7, 2013 at 10:38 AM, Felix GV <[EMAIL PROTECTED]>
> > wrote:
> > > > >
> > > > > > Thanks Jun!
> > > > > >
> > > > > > I hadn't been following the discussions regarding 0.8 and
> > replication
> > > > > for a
> > > > > > little while and this was a great post to refresh my memory and
> get
> > > up
> > > > to
> > > > > > speed on the current replication architecture's design.
> > > > > >
> > > > > > --
> > > > > > Felix
> > > > > >
> > > > > >
> > > > > > On Tue, Feb 5, 2013 at 2:21 PM, Jun Rao <[EMAIL PROTECTED]>
> wrote:
> > > > > >
> > > > > > > I just posted the following blog on Kafka replication. This may
> > > > answer
> > > > > > some
> > > > > > > of the questions that a few people have asked in the mailing
> list
> > > > > before.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://engineering.linkedin.com/kafka/intra-cluster-replication-apache-kafka
> > > > > > >
> > > > > > > Thanks,
> > > > > > >
> > > > > > > Jun
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>