Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - How to get topic offset infornation


Copy link to this message
-
Re: How to get topic offset infornation
Neha Narkhede 2013-04-30, 16:23
To get the earliest and latest offset for a particular topic
partition, you can use  the GetOffsetShell tool.

Thanks,
Neha

On Tue, Apr 30, 2013 at 7:52 AM, Snehalata Nagaje
<[EMAIL PROTECTED]> wrote:
> Hi Neha,
>
> I want to get largest and earliest offset available in topic, to check how
> many messages are consumed by client.
>
> I do not want to set auto.commit.enable to true. So that I can client API
> which is connecting to cluster will have better control over offset.
>
> And also is there any way I can fetch messages from kafka(in cluster mode)
> from particular offset?
>
>
> Thanks,
> Snehalata
>
> --------------------------------------------------
> From: "Neha Narkhede" <[EMAIL PROTECTED]>
> Sent: Tuesday, April 30, 2013 8:14 PM
> To: <[EMAIL PROTECTED]>
> Subject: Re: How to get topic offset infornation
>
>
>> Do you want offset information to check how much your consumer is lagging
>> or is there something else you are trying to achieve? You can use the
>> ConsumerOffsetChecker tool to see how many messages your high level
>> consumer is lagging behind by.
>>
>> Thanks,
>> Neha
>> On Apr 30, 2013 3:44 AM, "Snehalata Nagaje" <
>> [EMAIL PROTECTED]> wrote:
>>
>>> Hi,
>>>
>>> I am using kafka 0.8 version .
>>>
>>> I want to get topic offset information earlier and latest offset
>>> available
>>> in topic. I was able to get this using simpleconsumer API getOffseBefore.
>>>
>>> But I am using high level API to connect since I want to connect to kafka
>>> cluster using external zookeeper cluster.
>>>
>>> I have set up 3 kafka broker nodes with replication factor 2 and
>>> connected
>>> to zookeker cluster(3 -nodes).
>>>
>>> Can you please help with fetching offset information in cluster mode.
>>>
>>> Thanks,
>>> Snehalata
>>>
>>>
>>> Disclaimer:
>>> This e-mail may contain Privileged/Confidential information and is
>>> intended only for the individual(s) named. Any review, retransmission,
>>> dissemination or other use of, or taking of any action in reliance upon
>>> this information by persons or entities other than the intended recipient
>>> is prohibited. Please notify the sender, if you have received this e-mail
>>> by mistake and delete it from your system. Information in this message
>>> that
>>> does not relate to the official business of the company shall be
>>> understood
>>> as neither given nor endorsed by it. E-mail transmission cannot be
>>> guaranteed to be secure or error-free. The sender does not accept
>>> liability
>>> for any errors or omissions in the contents of this message which arise
>>> as
>>> a result of e-mail transmission. If verification is required please
>>> request
>>> a hard-copy version. Visit us at http://www.harbingergroup.com/
>>>
>>
> Disclaimer:
> This e-mail may contain Privileged/Confidential information and is intended
> only for the individual(s) named. Any review, retransmission, dissemination
> or other use of, or taking of any action in reliance upon this information
> by persons or entities other than the intended recipient is prohibited.
> Please notify the sender, if you have received this e-mail by mistake and
> delete it from your system. Information in this message that does not relate
> to the official business of the company shall be understood as neither given
> nor endorsed by it. E-mail transmission cannot be guaranteed to be secure or
> error-free. The sender does not accept liability for any errors or omissions
> in the contents of this message which arise as a result of e-mail
> transmission. If verification is required please request a hard-copy
> version. Visit us at http://www.harbingergroup.com/