Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> how to force a consumer to start at the beginning


+
Withers, Robert 2013-07-25, 16:12
+
James A. Robinson 2013-07-25, 17:25
+
Joe Stein 2013-07-25, 17:42
+
Rob Withers 2013-07-25, 17:49
+
Rob Withers 2013-07-25, 17:57
+
Florin Trofin 2013-07-25, 22:37
+
Rob Withers 2013-07-26, 02:36
Copy link to this message
-
Re: how to force a consumer to start at the beginning
Thanks, Jim.   I saw that in the 0.8 config as well.

I am trying to write a REST service that dumps all traffic in a given topic/partition.  The issue I seem to be facing now is the blocking API of the consumerIterator.  Is there any way we can ask whether the traffic is drained?  Perhaps a way to correlate topic/partition/thread/latestOffsetPerGroupId, maybe in the PartitionTopicInfo?  I see consumed and fetched offsets, but not a written offset.

How about a non-blocking API for the iterator: has that been discussed or spiked?

thanks,
rob

On Jul 25, 2013, at 11:24 AM, James A. Robinson <[EMAIL PROTECTED]> wrote:

> On Thu, Jul 25, 2013 at 9:11 AM, Withers, Robert
> <[EMAIL PROTECTED]> wrote:
>> We are creating a consumer with properties and I did not see a
>> property that screamed that it was to start at the beginning of a
>> topic.  Is there such a property?
>
> In v0.7, set 'autooffset.reset' to 'smallest'.
>
> Jim
>
> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> James A. Robinson                       [EMAIL PROTECTED]
> HighWire | Stanford University          http://highwire.stanford.edu/
> +1 650 7237294 (Work)                   +1 650 7259335 (Fax)
 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB