Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka, mail # user - Kafka log retentions


+
Ankit Jain 2013-04-01, 19:44
+
Philip OToole 2013-04-01, 19:50
+
Jason Rosenberg 2013-04-01, 20:00
Copy link to this message
-
Re: Kafka log retentions
Yonghui Zhao 2013-04-02, 03:45
A little question£¬ when old message log is deleted£¬ the start point offset
is changed. If a simple consumer seek a offset less than start point
offset, what will happen? Read all message from start?
2013/4/2 Jason Rosenberg <[EMAIL PROTECTED]>

> Essentially,
>
> There's a configuration property: log.retention.hours
>
> This determines the minimum time a message will remain available on the
> broker.  The default is 7 days.
>
> The kafka broker doesn't keep track of whether the message has been
> consumed or not (or how many times it has been consumed).
>
> It's up to individual consumers to keep track of which message offset
> they've consumed (which typically gets saved in zookeeper).
>
> Jason
>
>
> On Mon, Apr 1, 2013 at 12:50 PM, Philip O'Toole <[EMAIL PROTECTED]> wrote:
>
> > On Mon, Apr 1, 2013 at 12:27 PM, Ankit Jain <[EMAIL PROTECTED]>
> > wrote:
> >
> > > Hi All,
> > >
> > > Once the message is consumed by consumer, we want it to delete from
> > message
> > > broker as well.
> > >
> >
> > Kafka doesn't work this way. Read the design doc -- it's well written,
> and
> > should be read by anyone working with Kafka.
> >
> > http://kafka.apache.org/design.html
> >
> >
> >
> > >
> > > I was exploring the kafka configuration, but not sure which
> configuration
> > > would full my need.
> > >
> > > Guys, I need your help.
> > >
> > > Thanks ..
> > >
> > >
> > > --
> > > Thanks,
> > > Ankit Jain
> > >
> >
>

 
+
Philip OToole 2013-04-02, 03:50
+
Yonghui Zhao 2013-04-02, 03:55
+
Jun Rao 2013-04-02, 04:50
+
Tom Brown 2013-04-02, 05:46