I just posted the following blog on Kafka replication. This may answer some
of the questions that a few people have asked in the mailing list before.

http://engineering.linkedin.com/kafka/intra-cluster-replication-apache-kafka

Thanks,

Jun

 

Thanks Jun!

I hadn't been following the discussions regarding 0.8 and replication for a
little while and this was a great post to refresh my memory and get up to
speed on the current replication architecture's design.

--
Felix
On Tue, Feb 5, 2013 at 2:21 PM, Jun Rao <[EMAIL PROTECTED]> wrote:

> I just posted the following blog on Kafka replication. This may answer some
> of the questions that a few people have asked in the mailing list before.
>
>
> http://engineering.linkedin.com/kafka/intra-cluster-replication-apache-kafka
>
> Thanks,
>
> Jun
>

 

Same here. Thanks a lot Jun.

Regards,
Vaibhav

On Thu, Feb 7, 2013 at 10:38 AM, Felix GV <[EMAIL PROTECTED]> wrote:

> Thanks Jun!
>
> I hadn't been following the discussions regarding 0.8 and replication for a
> little while and this was a great post to refresh my memory and get up to
> speed on the current replication architecture's design.
>
> --
> Felix
>
>
> On Tue, Feb 5, 2013 at 2:21 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
>
> > I just posted the following blog on Kafka replication. This may answer
> some
> > of the questions that a few people have asked in the mailing list before.
> >
> >
> >
> http://engineering.linkedin.com/kafka/intra-cluster-replication-apache-kafka
> >
> > Thanks,
> >
> > Jun
> >
>

 

Same here, summary was need as we have a fairly large ecosystem of multiple
0.7.2 clusters and I am planning to test upgrade to 0.8.
However, one thing  creeping at the back of my mind regarding 0.8 is
something i have spotted in one thread few weeks ago namely that the
rebalance behaviour of producers is not as robust as in 0.7.x without
replication and i remeber there was no designed solution at the time - any
news here ? Basically our usecase doesn't require replication but logical
offsets and some other things introduced would solve some problems.
On Feb 7, 2013 7:11 PM, "Vaibhav Puranik" <[EMAIL PROTECTED]> wrote:

> Same here. Thanks a lot Jun.
>
> Regards,
> Vaibhav
>
> On Thu, Feb 7, 2013 at 10:38 AM, Felix GV <[EMAIL PROTECTED]> wrote:
>
> > Thanks Jun!
> >
> > I hadn't been following the discussions regarding 0.8 and replication
> for a
> > little while and this was a great post to refresh my memory and get up to
> > speed on the current replication architecture's design.
> >
> > --
> > Felix
> >
> >
> > On Tue, Feb 5, 2013 at 2:21 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
> >
> > > I just posted the following blog on Kafka replication. This may answer
> > some
> > > of the questions that a few people have asked in the mailing list
> before.
> > >
> > >
> > >
> >
> http://engineering.linkedin.com/kafka/intra-cluster-replication-apache-kafka
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> >
>

 

We have fixed this issue in 0.8. Withreplication factor 1, if the producer
doesn't care about partitioning by key, messages will be sent to partitions
that are currently available.

Thanks,

Jun

On Thu, Feb 7, 2013 at 3:11 PM, Michal Haris <[EMAIL PROTECTED]>wrote:

> Same here, summary was need as we have a fairly large ecosystem of multiple
> 0.7.2 clusters and I am planning to test upgrade to 0.8.
> However, one thing  creeping at the back of my mind regarding 0.8 is
> something i have spotted in one thread few weeks ago namely that the
> rebalance behaviour of producers is not as robust as in 0.7.x without
> replication and i remeber there was no designed solution at the time - any
> news here ? Basically our usecase doesn't require replication but logical
> offsets and some other things introduced would solve some problems.
> On Feb 7, 2013 7:11 PM, "Vaibhav Puranik" <[EMAIL PROTECTED]> wrote:
>
> > Same here. Thanks a lot Jun.
> >
> > Regards,
> > Vaibhav
> >
> > On Thu, Feb 7, 2013 at 10:38 AM, Felix GV <[EMAIL PROTECTED]> wrote:
> >
> > > Thanks Jun!
> > >
> > > I hadn't been following the discussions regarding 0.8 and replication
> > for a
> > > little while and this was a great post to refresh my memory and get up
> to
> > > speed on the current replication architecture's design.
> > >
> > > --
> > > Felix
> > >
> > >
> > > On Tue, Feb 5, 2013 at 2:21 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
> > >
> > > > I just posted the following blog on Kafka replication. This may
> answer
> > > some
> > > > of the questions that a few people have asked in the mailing list
> > before.
> > > >
> > > >
> > > >
> > >
> >
> http://engineering.linkedin.com/kafka/intra-cluster-replication-apache-kafka
> > > >
> > > > Thanks,
> > > >
> > > > Jun
> > > >
> > >
> >
>