Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - producer queue size


Copy link to this message
-
Re: producer queue size
Neha Narkhede 2012-11-27, 00:40
This will happen if you push data to the producer at a higher rate
than it is able to send to the server.
queue.size allows you to configure the size of the producer queue
(defaults to 10K). Also,
queue.enqueueTimeout.ms, if set to -1 will lead to blocking behaviour
instead of the producer throwing
QueueFullExceptions.

Thanks,
Neha

On Mon, Nov 26, 2012 at 4:06 PM, Joel Koshy <[EMAIL PROTECTED]> wrote:
> To use async, set producer.type to async ; The default queue size is 10000;
> and the default batch size is 200.
>
>
> On Mon, Nov 26, 2012 at 2:28 PM, Jamie Wang <[EMAIL PROTECTED]> wrote:
>
>> Hi, I am running the console demo comes with 0.7.2  in getting started
>> guide. All is working fine. Then I use stdio redirect a file of 30K lines
>> of messages into the producer and I received  an error "ERROR Event queue
>> is full of unsent messages, could not send event:..." and exception stack
>> shows:
>> Exception in thread "main" kafka.producer.async.QueueFullException: Event
>> queue is full of unsent messages, could not send event:...
>>
>> Is there a way in the produce.properties to configure the size of producer
>> queue?   I know the producer is running in synchronous mode. How do I solve
>> this problem or my usage is totally wrong?
>>
>> Thanks
>> Jamie
>>