Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Purgatory


Guozhang,
The documentation is not very clear.
Marc's response for producer purgatory makes sense.
I am not entirely clear on fetch purgatory.
How does broker use purgatory? Is it a temporary holding area? What happens
to the messages if purge interval is exceeded in case of either/both
producer and consumer? Are messages dropped in this case?
Thanks,
Priya
On Thu, Oct 31, 2013 at 2:47 PM, Guozhang Wang <[EMAIL PROTECTED]> wrote:

> Hello Priya,
>
> You can find the definitions of these two configs here:
>
> http://kafka.apache.org/documentation.html#brokerconfigs
>
> Guozhang
>
>
> On Thu, Oct 31, 2013 at 11:20 AM, Marc Labbe <[EMAIL PROTECTED]> wrote:
>
> > Hi Priya
> >
> > my understanding is producer requests will be delayed (and put in request
> > purgatory) only if your producer uses ack=-1. It will be in the purgatory
> > (delayed) until all brokers have acknowledged the messages to be
> > replicated. The documentation suggests to monitor the
> > ProducerRequestPurgatory size metrics , but it only applies if you're
> using
> > ack=-1, otherwise, this value will always be 0.
> >
> > For consumer requests, they'll be in purgatory (delayed) until the max
> > allowed time to respond has been reached, unless it has enough messages
> to
> > fill the buffer before that. The request will not end up in the purgatory
> > if you're making a blocking request (max wait <= 0).
> >
> > Not sure about the configuration interval though.
> >
> > marc
> >
> >
> > On Thu, Oct 31, 2013 at 12:41 PM, Priya Matpadi <
> > [EMAIL PROTECTED]
> > > wrote:
> >
> > > Hello,
> > > What is purgatory? I believe the following two properties relate to
> > > consumer and producer respectively.
> > > Could someone please explain the significance of these?
> > > fetch.purgatory.purge.interval.requests=100
> > > producer.purgatory.purge.interval.requests=100
> > >
> > > Thanks,
> > > Priya
> > >
> >
>
>
>
> --
> -- Guozhang
>