Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> PartitionOffsetRequestInfo only supports -1 and -2?


Copy link to this message
-
Re: PartitionOffsetRequestInfo only supports -1 and -2?
Jun,
    Regarding the arbitrary timestamp, I found that if I use the
GetOffsetShell<https://github.com/apache/kafka/blob/0.8/core/src/main/scala/kafka/tools/GetOffsetShell.scala>,
it works. But if I use the Java API ( requestInfo.put(topicAndPartition, new
 PartitionOffsetRequestInfo(SOME_LONG_TIMESTAMP, 1));), it only returns an
empty long[].

    I looked at the source of GetOffsetShell, the way the offset is
retrieved is slightly different from using the java api. Could that be a
bug?
Scala API:
val offsets = consumer.getOffsetsBefore(request).partitionErrorAndOffsets(
topicAndPartition).offsets

Java API:
consumer.getOffsetsBefore(request).offsets(topic, partition);
thanks,
Cal

On Tue, Jan 7, 2014 at 6:49 PM, Jun Rao <[EMAIL PROTECTED]> wrote:

> Joe,
>
> Yes, this is a bit confusing. We have 2 offset related requests. One is
> OffsetFetchRequest and another is OffsetRequest. The former is used to get
> the offset within a consumer group and the latter is to get the offset
> before a particular timestamp. And there is of course the FetchRequest,
> which fetches messages on a given offset.
>
> Thanks,
>
> Jun
>
>
> On Tue, Jan 7, 2014 at 9:00 AM, Joe Stein <[EMAIL PROTECTED]> wrote:
>
> > hmmm, that should be the "offset to fetch from", not sure why the
> variable
> > is called "time" =8^/  unless I am looking at something else from what
> you
> > are asking but this I think (another dev please confirm or explain why it
> > is time).
> >
> > case class PartitionOffsetRequestInfo(time: Long, maxNumOffsets: Int)
> >
> > should be
> >
> > case class PartitionOffsetRequestInfo(offsetToFetchFrom: Long,
> > maxNumOffsets: Int)
> >
> > which matches the Fetch API we have documented
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol
> >
> >
> > /*******************************************
> >  Joe Stein
> >  Founder, Principal Consultant
> >  Big Data Open Source Security LLC
> >  http://www.stealth.ly
> >  Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> > ********************************************/
> >
> >
> > On Tue, Jan 7, 2014 at 11:31 AM, Calvin Lei <[EMAIL PROTECTED]> wrote:
> >
> > > Does it support an long timestamp? Tried it and the resulting offset
> > > response contains an empty long array.
> > >
> > >
> > >
> > >
> > >
> > > --
> > > Sent from Gmail Mobile
> > >
> >
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB