Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Re: Batch size in producer

Copy link to this message
Re: Batch size in producer
> How big should be the batch size?

In production at Linkedin, we use a batch size of 200, which has worked
pretty well.
> What happens if producer client crashes before batch is full and messages
> were still sitting in producer queue?  Does it recover those messages when
> we restart the producer?

If the producer shuts down, it flushes out the messages in the internal
queue. If it crashes, it does not get a chance to do that and there is no
good alternative to this.
> Are these messages stored in memory or disk?

In memory.