I am looking at the reliability aspects of the system. I want to have some
1. Producer side spooling: If none of the broker is reachable by producer,
does the producer spool to disk and retry. I could not locate the code for
it. Can anybody point me the code if it does spooling ?
2. How folks integrate with the producers ? Do they directly stream messages
from the *embedded* client from producers or run a local agent which
forwards the messages to the broker ?
3. If a message has to be given to a particular partition (based on the
partitioning function) and the broker hosting that partition is down, then
what happen to that message. It can't be given to any other partition as it
will void the partitioning contract. Right ?
Hadoop and Avro Committer
Technology Platforms, InMobi
Disclaimer: Opinions expressed are my own and do not represent past or
Jun Rao 2011-10-14, 15:09
Sharad Agarwal 2011-10-15, 14:40