Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> producer queue size


Copy link to this message
-
Re: producer queue size
I wonder if this makes sense for the default. I agree that most people who
are choosing async would prefer to drop the message rather than block since
it is async. But I wonder if we should default to -1 anyway and let them
set this. The reason is that I think benchmarking and other cases always
run into this and it matches the behavior of other messaging systems.

-Jay
On Mon, Nov 26, 2012 at 4:40 PM, Neha Narkhede <[EMAIL PROTECTED]>wrote:

> This will happen if you push data to the producer at a higher rate
> than it is able to send to the server.
> queue.size allows you to configure the size of the producer queue
> (defaults to 10K). Also,
> queue.enqueueTimeout.ms, if set to -1 will lead to blocking behaviour
> instead of the producer throwing
> QueueFullExceptions.
>
> Thanks,
> Neha
>
> On Mon, Nov 26, 2012 at 4:06 PM, Joel Koshy <[EMAIL PROTECTED]> wrote:
> > To use async, set producer.type to async ; The default queue size is
> 10000;
> > and the default batch size is 200.
> >
> >
> > On Mon, Nov 26, 2012 at 2:28 PM, Jamie Wang <[EMAIL PROTECTED]>
> wrote:
> >
> >> Hi, I am running the console demo comes with 0.7.2  in getting started
> >> guide. All is working fine. Then I use stdio redirect a file of 30K
> lines
> >> of messages into the producer and I received  an error "ERROR Event
> queue
> >> is full of unsent messages, could not send event:..." and exception
> stack
> >> shows:
> >> Exception in thread "main" kafka.producer.async.QueueFullException:
> Event
> >> queue is full of unsent messages, could not send event:...
> >>
> >> Is there a way in the produce.properties to configure the size of
> producer
> >> queue?   I know the producer is running in synchronous mode. How do I
> solve
> >> this problem or my usage is totally wrong?
> >>
> >> Thanks
> >> Jamie
> >>
>