Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka, mail # user - Kafka Important Metrics


+
Vadim Keylis 2013-07-01, 18:21
+
Joel Koshy 2013-07-01, 20:26
+
Joel Koshy 2013-07-01, 20:27
+
Maxime Brugidou 2013-07-01, 21:57
Copy link to this message
-
Re: Kafka Important Metrics
Hanish Bansal 2013-07-02, 04:17
Hi Vadim,

It totally depends on your requirement. If you want all metrics in some
file then other option may be better otherwise the best way to monitor
kafka is through jmx.

Also list of metrics varies in different Kafka versions. I am assuming that
you are using recent Kafka release of 0.8.

Some important metrics in in Kafka-0.8 through JMX are:

BytesInCount
BytesOutCount
BytesInPerSecond
BytesOutPerSecond
FailedProduceRequestsCount
FailedProduceRequestsPerSecond
FailedFetchRequestsCount
FailedFetchRequestsPerSecond
MessageInCount
MessageInPerSecond

Hope it will be helpful for you.

On Mon, Jul 1, 2013 at 9:57 PM, Maxime Brugidou
<[EMAIL PROTECTED]>wrote:

> By the way, having an official contrib package with graphite, ganglia and
> other well-known reporters would be awesome so that not everyone has to
> write their own.
> On Jul 1, 2013 10:27 PM, "Joel Koshy" <[EMAIL PROTECTED]> wrote:
>
> > Also, there are several key metrics on the broker and client side - we
> > should compile a list and put it on a wiki. Can you file a jira for
> > this?
> >
> > On Mon, Jul 1, 2013 at 1:26 PM, Joel Koshy <[EMAIL PROTECTED]> wrote:
> > > CSVreporter is probably not an ideal fit for production monitoring -
> > > we use it for getting stats out of period system test runs.
> > >
> > > For production monitoring you are probably better off reading off JMX
> > > and feeding your monitoring system of choice. You can also write a
> > > custom metrics reporter and additionally implement
> > > KafkaMetricsReporter (see KafkaCSVMetricsReporter.scala for an
> > > example) and plug it in the KafkaConfig. Your custom metrics reporter
> > > can directly feed into your monitoring system. E.g., if you use
> > > graphite, you can write a KafkaGraphiteReporter that wraps the
> > > GraphiteReporter that coda hale metrics provides.
> > >
> > > Joel
> > >
> > > On Mon, Jul 1, 2013 at 11:21 AM, Vadim Keylis <[EMAIL PROTECTED]>
> > wrote:
> > >> Good morning. What is the best way to monitor kafka through jmx or by
> > >> enabling kafka.csv.metrics.reporter.enabled?
> > >> What are the important metrics in JMX to watch for and/or graph?
> > >> What are the important metrics in csv files to watch for and/or graph?
> > >>
> > >> Thanks,
> > >> Vadim
> >
>

--
*Thanks & Regards*
*Hanish Bansal*

 
+
Cosmin Lehene 2013-07-03, 15:23
+
David DeMaagd 2013-07-03, 16:08
+
Cosmin Lehene 2013-07-03, 16:24
+
Vadim Keylis 2013-07-03, 14:48