Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> metrics with generation specific names


+
Jason Rosenberg 2013-10-23, 03:56
+
Jun Rao 2013-10-23, 04:29
Copy link to this message
-
Re: metrics with generation specific names
filed: https://issues.apache.org/jira/browse/KAFKA-1100
On Wed, Oct 23, 2013 at 12:28 AM, Jun Rao <[EMAIL PROTECTED]> wrote:

> Yes, those metrics names could be simplified. Could you file a jira?
>
> Thanks,
>
> Jun
>
>
> On Tue, Oct 22, 2013 at 8:55 PM, Jason Rosenberg <[EMAIL PROTECTED]> wrote:
>
> > I've noticed that there are several metrics that seem useful for
> monitoring
> > overtime, but which contain generational timestamps in the metric name.
> >
> > We are using yammer metrics libraries to send metrics data in a
> background
> > thread every 10 seconds (to kafka actually), and then they eventually end
> > up in a metrics database (graphite, opentsdb).  The metrics then get
> > graphed via UI, and we can see metrics going way back, etc.
> >
> > Unfortunately, many of the metrics coming from kafka seem to have metric
> > names that change any time the server or consumer is restarted, which
> makes
> > it hard to easily create graphs over long periods of time (spanning app
> > restarts).
> >
> > For example:
> >
> > names like:
> > kafka.consumer.FetchRequestAndResponseMetrics....
> >
> >
> square-1371718712833-e9bb4d10-0-508818741-AllBrokersFetchRequestRateAndTimeMs<
> > https://graphite.vip.sjc1.square/composer/?#>
> >
> > or:
> > kafka.consumer.ZookeeperConsumerConnector...topicName.....
> > square-1373476779391-78aa2e83-0-FetchQueueSize<
> > https://graphite.vip.sjc1.square/composer/?#>
> >
> > In our staging environment, we have our servers on regular auto-deploy
> > cycles (they restart every few hours).  So just not longitudinally usable
> > to have metric names constantly changing like this.
> >
> > Is there something that can easily be done?  Is it really necessary to
> have
> > so much cryptic info in the metric name?
> >
> > Jason
> >
>