Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Re: Batch size in producer


Copy link to this message
-
Re: Batch size in producer
> How big should be the batch size?
>

In production at Linkedin, we use a batch size of 200, which has worked
pretty well.
> What happens if producer client crashes before batch is full and messages
> were still sitting in producer queue?  Does it recover those messages when
> we restart the producer?
>

If the producer shuts down, it flushes out the messages in the internal
queue. If it crashes, it does not get a chance to do that and there is no
good alternative to this.
> Are these messages stored in memory or disk?
>

In memory.

Thanks,
Neha