Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> why was auto.offset.reset default changed (so late in the game)?


Copy link to this message
-
why was auto.offset.reset default changed (so late in the game)?
I'm wondering why the default setting for auto.offset.reset in the
ConsumerConfig class was changed from 'smallest' to 'largest', so late in
the game (looks like a commit on June 3 changed the default).  This is an
extremely major change, I should think.  Consumers now by default only get
messages newer than when the consumer starts?

What use cases are there for that?  I can think of one off cases, where you
just want to start consuming the latest feed, etc., to bootstrap things.
 But in the normal case, where you want to take a consumer down for an
update, and bring it back up, you'd always be losing messages in that case.

The default of the old (now renamed) "autooffset.reset" property in 0.7.2
is "smallest", so this is a major change.

Sadly, this one change caused me many hours of consternation with some
broken tests (e.g. KAFKA-945).  I don't see any mention of this change
listed in any messages to the group, etc.

It might be good to have a configuration migration page outlining changes
from 0.7.2.  This change is particularly difficult since it is in a default
setting that in most cases people had been using the default (and now will
override the default in most cases).

Jason

 
+
Jun Rao 2013-06-18, 19:50
+
Jason Rosenberg 2013-06-18, 20:28
+
Jun Rao 2013-06-18, 23:05