Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> PartitionOffsetRequestInfo only supports -1 and -2?


Copy link to this message
-
Re: PartitionOffsetRequestInfo only supports -1 and -2?
Jun,
    Regarding the arbitrary timestamp, I found that if I use the
GetOffsetShell<https://github.com/apache/kafka/blob/0.8/core/src/main/scala/kafka/tools/GetOffsetShell.scala>,
it works. But if I use the Java API ( requestInfo.put(topicAndPartition, new
 PartitionOffsetRequestInfo(SOME_LONG_TIMESTAMP, 1));), it only returns an
empty long[].

    I looked at the source of GetOffsetShell, the way the offset is
retrieved is slightly different from using the java api. Could that be a
bug?
Scala API:
val offsets = consumer.getOffsetsBefore(request).partitionErrorAndOffsets(
topicAndPartition).offsets

Java API:
consumer.getOffsetsBefore(request).offsets(topic, partition);
thanks,
Cal

On Tue, Jan 7, 2014 at 6:49 PM, Jun Rao <[EMAIL PROTECTED]> wrote:

> Joe,
>
> Yes, this is a bit confusing. We have 2 offset related requests. One is
> OffsetFetchRequest and another is OffsetRequest. The former is used to get
> the offset within a consumer group and the latter is to get the offset
> before a particular timestamp. And there is of course the FetchRequest,
> which fetches messages on a given offset.
>
> Thanks,
>
> Jun
>
>
> On Tue, Jan 7, 2014 at 9:00 AM, Joe Stein <[EMAIL PROTECTED]> wrote:
>
> > hmmm, that should be the "offset to fetch from", not sure why the
> variable
> > is called "time" =8^/  unless I am looking at something else from what
> you
> > are asking but this I think (another dev please confirm or explain why it
> > is time).
> >
> > case class PartitionOffsetRequestInfo(time: Long, maxNumOffsets: Int)
> >
> > should be
> >
> > case class PartitionOffsetRequestInfo(offsetToFetchFrom: Long,
> > maxNumOffsets: Int)
> >
> > which matches the Fetch API we have documented
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol
> >
> >
> > /*******************************************
> >  Joe Stein
> >  Founder, Principal Consultant
> >  Big Data Open Source Security LLC
> >  http://www.stealth.ly
> >  Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> > ********************************************/
> >
> >
> > On Tue, Jan 7, 2014 at 11:31 AM, Calvin Lei <[EMAIL PROTECTED]> wrote:
> >
> > > Does it support an long timestamp? Tried it and the resulting offset
> > > response contains an empty long array.
> > >
> > >
> > >
> > >
> > >
> > > --
> > > Sent from Gmail Mobile
> > >
> >
>