Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> producer queue size


Copy link to this message
-
Re: producer queue size
This will happen if you push data to the producer at a higher rate
than it is able to send to the server.
queue.size allows you to configure the size of the producer queue
(defaults to 10K). Also,
queue.enqueueTimeout.ms, if set to -1 will lead to blocking behaviour
instead of the producer throwing
QueueFullExceptions.

Thanks,
Neha

On Mon, Nov 26, 2012 at 4:06 PM, Joel Koshy <[EMAIL PROTECTED]> wrote:
> To use async, set producer.type to async ; The default queue size is 10000;
> and the default batch size is 200.
>
>
> On Mon, Nov 26, 2012 at 2:28 PM, Jamie Wang <[EMAIL PROTECTED]> wrote:
>
>> Hi, I am running the console demo comes with 0.7.2  in getting started
>> guide. All is working fine. Then I use stdio redirect a file of 30K lines
>> of messages into the producer and I received  an error "ERROR Event queue
>> is full of unsent messages, could not send event:..." and exception stack
>> shows:
>> Exception in thread "main" kafka.producer.async.QueueFullException: Event
>> queue is full of unsent messages, could not send event:...
>>
>> Is there a way in the produce.properties to configure the size of producer
>> queue?   I know the producer is running in synchronous mode. How do I solve
>> this problem or my usage is totally wrong?
>>
>> Thanks
>> Jamie
>>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB