Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka, mail # dev - Reliability aspects


Copy link to this message
-
Reliability aspects
Sharad Agarwal 2011-10-14, 05:58
I am looking at the reliability aspects of the system. I want to have some
clarity on:

1. Producer side spooling: If none of the broker is reachable by producer,
does the producer spool to disk and retry. I could not locate the code for
it. Can anybody point me the code if it does spooling ?

2. How folks integrate with the producers ? Do they directly stream messages
from the *embedded* client from producers or run a local agent which
forwards the messages to the broker ?

3. If a message has to be given to a particular partition (based on the
partitioning function) and the broker hosting that partition is down, then
what happen to that message. It can't be given to any other partition as it
will void the partitioning contract. Right ?
Thanks
Sharad Agarwal
Hadoop and Avro Committer
Technology Platforms, InMobi
Disclaimer: Opinions expressed are my own and do not represent past or
present employers.
+
Jun Rao 2011-10-14, 15:09
+
Sharad Agarwal 2011-10-15, 14:40