Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - Kafka log retentions


Copy link to this message
-
Re: Kafka log retentions
Jason Rosenberg 2013-04-01, 20:00
Essentially,

There's a configuration property: log.retention.hours

This determines the minimum time a message will remain available on the
broker.  The default is 7 days.

The kafka broker doesn't keep track of whether the message has been
consumed or not (or how many times it has been consumed).

It's up to individual consumers to keep track of which message offset
they've consumed (which typically gets saved in zookeeper).

Jason
On Mon, Apr 1, 2013 at 12:50 PM, Philip O'Toole <[EMAIL PROTECTED]> wrote:

> On Mon, Apr 1, 2013 at 12:27 PM, Ankit Jain <[EMAIL PROTECTED]>
> wrote:
>
> > Hi All,
> >
> > Once the message is consumed by consumer, we want it to delete from
> message
> > broker as well.
> >
>
> Kafka doesn't work this way. Read the design doc -- it's well written, and
> should be read by anyone working with Kafka.
>
> http://kafka.apache.org/design.html
>
>
>
> >
> > I was exploring the kafka configuration, but not sure which configuration
> > would full my need.
> >
> > Guys, I need your help.
> >
> > Thanks ..
> >
> >
> > --
> > Thanks,
> > Ankit Jain
> >
>