Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> How to get topic offset infornation


Copy link to this message
-
Re: How to get topic offset infornation
To get the earliest and latest offset for a particular topic
partition, you can use  the GetOffsetShell tool.

Thanks,
Neha

On Tue, Apr 30, 2013 at 7:52 AM, Snehalata Nagaje
<[EMAIL PROTECTED]> wrote:
> Hi Neha,
>
> I want to get largest and earliest offset available in topic, to check how
> many messages are consumed by client.
>
> I do not want to set auto.commit.enable to true. So that I can client API
> which is connecting to cluster will have better control over offset.
>
> And also is there any way I can fetch messages from kafka(in cluster mode)
> from particular offset?
>
>
> Thanks,
> Snehalata
>
> --------------------------------------------------
> From: "Neha Narkhede" <[EMAIL PROTECTED]>
> Sent: Tuesday, April 30, 2013 8:14 PM
> To: <[EMAIL PROTECTED]>
> Subject: Re: How to get topic offset infornation
>
>
>> Do you want offset information to check how much your consumer is lagging
>> or is there something else you are trying to achieve? You can use the
>> ConsumerOffsetChecker tool to see how many messages your high level
>> consumer is lagging behind by.
>>
>> Thanks,
>> Neha
>> On Apr 30, 2013 3:44 AM, "Snehalata Nagaje" <
>> [EMAIL PROTECTED]> wrote:
>>
>>> Hi,
>>>
>>> I am using kafka 0.8 version .
>>>
>>> I want to get topic offset information earlier and latest offset
>>> available
>>> in topic. I was able to get this using simpleconsumer API getOffseBefore.
>>>
>>> But I am using high level API to connect since I want to connect to kafka
>>> cluster using external zookeeper cluster.
>>>
>>> I have set up 3 kafka broker nodes with replication factor 2 and
>>> connected
>>> to zookeker cluster(3 -nodes).
>>>
>>> Can you please help with fetching offset information in cluster mode.
>>>
>>> Thanks,
>>> Snehalata
>>>
>>>
>>> Disclaimer:
>>> This e-mail may contain Privileged/Confidential information and is
>>> intended only for the individual(s) named. Any review, retransmission,
>>> dissemination or other use of, or taking of any action in reliance upon
>>> this information by persons or entities other than the intended recipient
>>> is prohibited. Please notify the sender, if you have received this e-mail
>>> by mistake and delete it from your system. Information in this message
>>> that
>>> does not relate to the official business of the company shall be
>>> understood
>>> as neither given nor endorsed by it. E-mail transmission cannot be
>>> guaranteed to be secure or error-free. The sender does not accept
>>> liability
>>> for any errors or omissions in the contents of this message which arise
>>> as
>>> a result of e-mail transmission. If verification is required please
>>> request
>>> a hard-copy version. Visit us at http://www.harbingergroup.com/
>>>
>>
> Disclaimer:
> This e-mail may contain Privileged/Confidential information and is intended
> only for the individual(s) named. Any review, retransmission, dissemination
> or other use of, or taking of any action in reliance upon this information
> by persons or entities other than the intended recipient is prohibited.
> Please notify the sender, if you have received this e-mail by mistake and
> delete it from your system. Information in this message that does not relate
> to the official business of the company shall be understood as neither given
> nor endorsed by it. E-mail transmission cannot be guaranteed to be secure or
> error-free. The sender does not accept liability for any errors or omissions
> in the contents of this message which arise as a result of e-mail
> transmission. If verification is required please request a hard-copy
> version. Visit us at http://www.harbingergroup.com/

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB