Have you tried configuring the num.retries option on the log4j producer ?
When there is a temporary network glitch, it will retry sending the
messages instead of losing them.
On Thursday, March 14, 2013, Ryan Chan wrote:
> We are currently using the Log4j appender to write to the Kafka (on
> machine on another datacenter), but problem occur when the network is
> interrupted as the message is stored in memory.
> Are there any local forwarder/client or similar solutions that can be
> installed on the localhost and take care of the message forwarding?