Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # dev >> Initializing fetch offsets


+
David Arthur 2013-02-21, 02:32
+
Neha Narkhede 2013-02-21, 05:13
+
David Arthur 2013-02-21, 13:04
Copy link to this message
-
Re: Initializing fetch offsets
In the old protocol description the -1 and -2 values are explicitly
mentioned.
https://cwiki.apache.org/confluence/display/KAFKA/Wire+Format

It looks like it had been left out of the new 0.8 docs, however I just
added it.
https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-OffsetAPI

Hopefully the docs are more helpful now. Feel free to ask for further
clarifications.
On 02/21/2013 05:04 AM, David Arthur wrote:
> Yes, I think that section of the protocol guide could be improved.
> Specifically, it would be useful to explain what the " Time" and
> "MaxNumberOfOffsets" parameters are used for, as well documenting these
> two magic numbers (-1 and -2).
>
> -David
>
> On 2/21/13 12:13 AM, Neha Narkhede wrote:
>> David,
>>
>> The API that you are looking for is OffsetRequest. If you pass in -2 as the
>> value for Time, it will get you the earliest offset available for that
>> partition. If you pass in -1, it will get you the latest offset.
>>
>> You can find more information about the OffsetRequest and OffsetResponse
>> APIs here -
>> https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-OffsetAPI
>>
>> Please let us know if that description can be improved.
>>
>> Thanks,
>> Neha
>>
>>
>> On Wed, Feb 20, 2013 at 6:31 PM, David Arthur <[EMAIL PROTECTED]> wrote:
>>
>>> Is there currently a way to get the latest offset (or close to the latest)
>>> for a partition? Is this covered by the Offset API, and if so could someone
>>> explain this API a bit?
>>>
>>> I'd like to write a consumer that can start from 0 (like unix cat) or
>>> start from the most recent (like unix tail).
>>>
>>> Thanks!
>>> -David
>>>
--
*Colin Blower*
/Software Engineer/
Barracuda Networks Inc.
+1 408-342-5576 (o)

 
+
Neha Narkhede 2013-02-21, 20:40
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB