Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> What happens if one broker goes down


+
Chitra Raveendran 2013-05-17, 05:21
+
Neha Narkhede 2013-05-17, 13:27
+
Chitra Raveendran 2013-05-20, 06:32
Copy link to this message
-
Re: What happens if one broker goes down
0.7.2 does not support replication. So when a broker goes down, there can
be some data loss. If you are ok with duplicates, you can configure the
producer side retries to be higher.

Thanks,
Neha
On May 19, 2013 11:32 PM, "Chitra Raveendran" <
[EMAIL PROTECTED]> wrote:

> HI
>
> When my broker went down, my producer just stopped reading the file giving
> an exception saying "connection refused to kafka broker".
> I tried overcoming the exception by giving a try and catch ! BUT I'M SEEING
> DATA LOSS. What am I doing wrong ??? Please help
>
> For reference :
> I'm using kafka 0.7.2 since my consumer is a storm spout and i read that
> storm is most compatible with kafka 0.7.2.
> IT DOESNT SUPPORT REPLICATION RIGHT?
>
>
> On Fri, May 17, 2013 at 6:56 PM, Neha Narkhede <[EMAIL PROTECTED]
> >wrote:
>
> > You can read the high level design of kafka replication here
> > http://www.slideshare.net/junrao/kafka-replication-apachecon2013
> >
> > Generally if your replication factor is more than 1 you shouldn't see
> data
> > loss in your test. When a broker fails, the producer will get an
> exception
> > and it will retry.
> >
> > Thanks,
> > Neha
> > On May 16, 2013 10:21 PM, "Chitra Raveendran" <
> > [EMAIL PROTECTED]> wrote:
> >
> > > Hi
> > >
> > > I was just doing some benchmarking with a 3node cluster. If one broker
> > goes
> > > down , what happens? Does the producer go down ? that's what happened
> in
> > my
> > > case.
> > > Is the data lost? Or does it get distributed amongst the other brokers?
> > >
> > > --
> > > Chitra Raveendran
> > > Data Scientist
> > > *Flutura |  Decision Sciences & Analytics*
> > > mail  : [EMAIL PROTECTED]
> > >
> >
>
>
>
> --
> Chitra Raveendran
> Data Scientist
> *Flutura Business Solutions Pvt. Ltd*
> Tel     : +918197563660
> email  : [EMAIL PROTECTED]
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB