-Re: Metrics: via Broker vs. Producer vs. Consumer
Scott Clasen 2013-07-24, 16:25
Valid assumptions IMO
Sent from my iPhone
On Jul 24, 2013, at 8:22 AM, Jay Kreps <[EMAIL PROTECTED]> wrote:
> Yeah all our monitoring is just of the local process (basically just a
> counter exposed through yammer metrics which support jmx and other
> outputs). If I understand what you want instead of having a counter that
> tracks, say, produce requests per second for a single broker you want one
> that covers the whole cluster. Obviously this would require collecting the
> local count and aggregating across all the brokers.
> Our assumption is that you already have a separate monitoring system which
> can slurp all these up, aggregate them, graph them, and alert off them.
> There are a number of open source thingies like this and I think most
> bigger shops have something they use. Our assumption is that trying to do a
> kafka-specific monitoring system wouldn't work for most people because they
> are wedded to their current setup and just want to integrate with that.
> I'm not sure how valid any of those assumptions actually are.
> On Wed, Jul 24, 2013 at 7:29 AM, Otis Gospodnetic <
> [EMAIL PROTECTED]> wrote:
>> I was looking at
>> and noticed there is no information about which metrics are available
>> in which process/JVM/JMX.
>> Some are available in the Broker process, but some are only available
>> from the JVM running Consumer and some only from the JVM running
>> Producer. And yet some Producer and Consumer metrics are, I *believe*
>> available from Broker's JMX.
>> Would it be possible for somebody in the know to mark the metrics in
>> so one can tell where to get it?
>> Also, why is it that the Broker process doesn't have *all* metrics,
>> including Producer and Consumer one? Is that because there can be N
>> Brokers and each P or C talk to one Broker at a time and thus there is
>> no single process/JMX that can know *all* stats for *all* Brokers and
>> for *all* Ps and Cs?
>> Thank you!
>> Performance Monitoring -- http://sematext.com/spm
>> Solr & ElasticSearch Support -- http://sematext.com/