Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> metrics with generation specific names


+
Jason Rosenberg 2013-10-23, 03:56
+
Jun Rao 2013-10-23, 04:29
Copy link to this message
-
Re: metrics with generation specific names
filed: https://issues.apache.org/jira/browse/KAFKA-1100
On Wed, Oct 23, 2013 at 12:28 AM, Jun Rao <[EMAIL PROTECTED]> wrote:

> Yes, those metrics names could be simplified. Could you file a jira?
>
> Thanks,
>
> Jun
>
>
> On Tue, Oct 22, 2013 at 8:55 PM, Jason Rosenberg <[EMAIL PROTECTED]> wrote:
>
> > I've noticed that there are several metrics that seem useful for
> monitoring
> > overtime, but which contain generational timestamps in the metric name.
> >
> > We are using yammer metrics libraries to send metrics data in a
> background
> > thread every 10 seconds (to kafka actually), and then they eventually end
> > up in a metrics database (graphite, opentsdb).  The metrics then get
> > graphed via UI, and we can see metrics going way back, etc.
> >
> > Unfortunately, many of the metrics coming from kafka seem to have metric
> > names that change any time the server or consumer is restarted, which
> makes
> > it hard to easily create graphs over long periods of time (spanning app
> > restarts).
> >
> > For example:
> >
> > names like:
> > kafka.consumer.FetchRequestAndResponseMetrics....
> >
> >
> square-1371718712833-e9bb4d10-0-508818741-AllBrokersFetchRequestRateAndTimeMs<
> > https://graphite.vip.sjc1.square/composer/?#>
> >
> > or:
> > kafka.consumer.ZookeeperConsumerConnector...topicName.....
> > square-1373476779391-78aa2e83-0-FetchQueueSize<
> > https://graphite.vip.sjc1.square/composer/?#>
> >
> > In our staging environment, we have our servers on regular auto-deploy
> > cycles (they restart every few hours).  So just not longitudinally usable
> > to have metric names constantly changing like this.
> >
> > Is there something that can easily be done?  Is it really necessary to
> have
> > so much cryptic info in the metric name?
> >
> > Jason
> >
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB