Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka, mail # dev - Initializing fetch offsets


+
David Arthur 2013-02-21, 02:32
+
Neha Narkhede 2013-02-21, 05:13
+
David Arthur 2013-02-21, 13:04
Copy link to this message
-
Re: Initializing fetch offsets
Colin Blower 2013-02-21, 16:54
In the old protocol description the -1 and -2 values are explicitly
mentioned.
https://cwiki.apache.org/confluence/display/KAFKA/Wire+Format

It looks like it had been left out of the new 0.8 docs, however I just
added it.
https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-OffsetAPI

Hopefully the docs are more helpful now. Feel free to ask for further
clarifications.
On 02/21/2013 05:04 AM, David Arthur wrote:
> Yes, I think that section of the protocol guide could be improved.
> Specifically, it would be useful to explain what the " Time" and
> "MaxNumberOfOffsets" parameters are used for, as well documenting these
> two magic numbers (-1 and -2).
>
> -David
>
> On 2/21/13 12:13 AM, Neha Narkhede wrote:
>> David,
>>
>> The API that you are looking for is OffsetRequest. If you pass in -2 as the
>> value for Time, it will get you the earliest offset available for that
>> partition. If you pass in -1, it will get you the latest offset.
>>
>> You can find more information about the OffsetRequest and OffsetResponse
>> APIs here -
>> https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-OffsetAPI
>>
>> Please let us know if that description can be improved.
>>
>> Thanks,
>> Neha
>>
>>
>> On Wed, Feb 20, 2013 at 6:31 PM, David Arthur <[EMAIL PROTECTED]> wrote:
>>
>>> Is there currently a way to get the latest offset (or close to the latest)
>>> for a partition? Is this covered by the Offset API, and if so could someone
>>> explain this API a bit?
>>>
>>> I'd like to write a consumer that can start from 0 (like unix cat) or
>>> start from the most recent (like unix tail).
>>>
>>> Thanks!
>>> -David
>>>
--
*Colin Blower*
/Software Engineer/
Barracuda Networks Inc.
+1 408-342-5576 (o)

 
+
Neha Narkhede 2013-02-21, 20:40