Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Understanding the min fetch rate metric


Copy link to this message
-
Re: Understanding the min fetch rate metric
And, I assume the 'minFetchRate' refers to the lowest fetch rate over
all the fetchers (1 per broker) that a consumer has. I see....

On Tue, Dec 24, 2013 at 8:58 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
> There is one fetcher per broker, which is responsible for fetching messages
> in all consumed topics whose leader is on that broker. We have seen a
> fetcher being killed by a bug in Kafka. Also, if the broker is slow (e.g.
> due to I/O contention), the fetch rate could also be slower than expected.
>
> Thanks,
>
> Jun
>
>
> On Tue, Dec 24, 2013 at 12:48 PM, Jason Rosenberg <[EMAIL PROTECTED]> wrote:
>
>> Thanks Jun,
>>
>> I'm still not 100% clear on this.  Is the min-fetch rate per topic, or
>> is it the lowest fetch rate over all topics?  Or is it not topic
>> specific at all.
>>
>> What would cause this to slow down?  Or is it more a measure of the
>> consumer's rate at processing messages and ability fetching new
>> messages (regardless of topic)?
>>
>> If we have some topics with no data at all, that match the consumer's
>> topic filter, should we expect to always see a min fetch rate of zero
>> (since at least one topic really has no messages)?
>>
>> Jason
>>
>> On Tue, Dec 24, 2013 at 1:10 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
>> > I updated http://kafka.apache.org/documentation.html#monitoring
>> >
>> > Thanks,
>> >
>> > Jun
>> >
>> >
>> > On Mon, Dec 23, 2013 at 10:51 PM, Jason Rosenberg <[EMAIL PROTECTED]>
>> wrote:
>> >
>> >> I'm realizing I'm not quite sure what the 'min fetch rate' metrics is
>> >> indicating, for consumers.  Can someone offer an explanation?
>> >>
>> >> Is it related to the 'max lag' metric?
>> >>
>> >> Jason
>> >>
>>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB