Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - Understanding the min fetch rate metric


Copy link to this message
-
Re: Understanding the min fetch rate metric
Jason Rosenberg 2013-12-26, 22:13
And, I assume the 'minFetchRate' refers to the lowest fetch rate over
all the fetchers (1 per broker) that a consumer has. I see....

On Tue, Dec 24, 2013 at 8:58 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
> There is one fetcher per broker, which is responsible for fetching messages
> in all consumed topics whose leader is on that broker. We have seen a
> fetcher being killed by a bug in Kafka. Also, if the broker is slow (e.g.
> due to I/O contention), the fetch rate could also be slower than expected.
>
> Thanks,
>
> Jun
>
>
> On Tue, Dec 24, 2013 at 12:48 PM, Jason Rosenberg <[EMAIL PROTECTED]> wrote:
>
>> Thanks Jun,
>>
>> I'm still not 100% clear on this.  Is the min-fetch rate per topic, or
>> is it the lowest fetch rate over all topics?  Or is it not topic
>> specific at all.
>>
>> What would cause this to slow down?  Or is it more a measure of the
>> consumer's rate at processing messages and ability fetching new
>> messages (regardless of topic)?
>>
>> If we have some topics with no data at all, that match the consumer's
>> topic filter, should we expect to always see a min fetch rate of zero
>> (since at least one topic really has no messages)?
>>
>> Jason
>>
>> On Tue, Dec 24, 2013 at 1:10 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
>> > I updated http://kafka.apache.org/documentation.html#monitoring
>> >
>> > Thanks,
>> >
>> > Jun
>> >
>> >
>> > On Mon, Dec 23, 2013 at 10:51 PM, Jason Rosenberg <[EMAIL PROTECTED]>
>> wrote:
>> >
>> >> I'm realizing I'm not quite sure what the 'min fetch rate' metrics is
>> >> indicating, for consumers.  Can someone offer an explanation?
>> >>
>> >> Is it related to the 'max lag' metric?
>> >>
>> >> Jason
>> >>
>>