Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> quieting HBase metrics


+
Ron Echeverri 2013-09-18, 23:17
+
Arati Patro 2013-09-23, 11:10
+
Otis Gospodnetic 2013-09-24, 02:59
+
Bing Jiang 2013-09-24, 03:29
+
Ron Echeverri 2013-09-24, 04:55
Copy link to this message
-
Re: quieting HBase metrics
hi, Ron.
Indeed, I have rewrote the implementation of
org.apache.hadoop.hbase.regionserver.metrics.RegionServerMetrics
org.apache.hadoop.hbase.regionserver.metrics.RegionServerDynamicMetrics
There are major changes as below:
1) change the context name for hbase to separate name for each other;
like hadoop-metrics.properties:
hbase-rsdm respresents DynamicMetrics
and hbase-rs represents RegionServerMetrics.

2) add a filterMapper in Metrics, when it updates to ganglia or other
metrics collector.
so in our implementation, it will fetch the filter from
hadoop-metrics.properties, and emit metrics as to settings.

3) In order to support dynamic add/remove filter in
hadoop-metrics.properties, we alternatively add a rpc to restart metrics
system in RegionServer without restarting rs.

Tks
2013/9/24 Ron Echeverri <[EMAIL PROTECTED]>

> Thanks, Bing, but i still don't understand *how* to configure that.  Could
> you share your hadoop-metrics.properties file, or any other files you use
> for this?
>
> rone
>
>
> On Mon, Sep 23, 2013 at 8:29 PM, Bing Jiang <[EMAIL PROTECTED]
> >wrote:
>
> > hbase metrics contains different types:RegionServerMetrics and
> > RegionServerDynamicMetrics.
> > And there are more metrics emitted from RegionServerDynamicMetrics,
> > especially in case that there are more regions in RS.
> > RegionServerMetrics and RSDM share one metrics context, which makes the
> > control become difficult.
> >
> > In our production, we add a metrics filter in MetricsContext, which can
> be
> > configured as to our needs, which will reduce the amount of metrics, and
> > take a good effect.
> >
> >
> >
> >
> > 2013/9/24 Otis Gospodnetic <[EMAIL PROTECTED]>
> >
> >> Me, too!
> >>
> >> Otis
> >> --
> >> HBase Performance Monitoring -- http://sematext.com/spm
> >>
> >>
> >>
> >> On Mon, Sep 23, 2013 at 7:10 AM, Arati Patro <[EMAIL PROTECTED]>
> >> wrote:
> >> > Hi All,
> >> >
> >> >
> >> > I have the same query. Tried searching for a solution on the web but
> >> found
> >> > nothing very helpful. Is there some way I could disable metrics for
> >> every
> >> > table in HBase from being sent to Ganglia? Or just any information on
> >> how
> >> > to configure HBase to emit fewer metrics?
> >> >
> >> >
> >> > Thanks,
> >> > Arati Patro
> >> >
> >> >
> >> > On Thu, Sep 19, 2013 at 4:47 AM, Ron Echeverri <
> [EMAIL PROTECTED]
> >> >wrote:
> >> >
> >> >> In http://hbase.apache.org/book/hbase_metrics.html we see
> >> >>
> >> >> """
> >> >> 15.4.2. Warning To Ganglia Users
> >> >>
> >> >> Warning to Ganglia Users: by default, HBase will emit a LOT of
> metrics
> >> per
> >> >> RegionServer which may swamp your installation. Options include
> either
> >> >> increasing Ganglia server capacity, or configuring HBase to emit
> fewer
> >> >> metrics.
> >> >> """
> >> >>
> >> >> However, although there is documentation to help set up Ganglia
> >> metrics for
> >> >> HBase, there doesn't seem to be any that shows how to make HBase emit
> >> fewer
> >> >> metrics.  Could someone lend me a hand?  I have increased the period
> in
> >> >> hadoop-metrics.properties but that has proven insufficient.
> >> >>
> >> >> thanks
> >> >> rone
> >> >>
> >>
> >
> >
> >
> > --
> > Bing Jiang
> > weibo: http://weibo.com/jiangbinglover
> > BLOG: www.binospace.com
> > BLOG: http://blog.sina.com.cn/jiangbinglover
> > Focus on distributed computing, HDFS/HBase
> >
> >
> > 2013/9/24 Otis Gospodnetic <[EMAIL PROTECTED]>
> >
> >> Me, too!
> >>
> >> Otis
> >> --
> >> HBase Performance Monitoring -- http://sematext.com/spm
> >>
> >>
> >>
> >> On Mon, Sep 23, 2013 at 7:10 AM, Arati Patro <[EMAIL PROTECTED]>
> >> wrote:
> >> > Hi All,
> >> >
> >> >
> >> > I have the same query. Tried searching for a solution on the web but
> >> found
> >> > nothing very helpful. Is there some way I could disable metrics for
> >> every
> >> > table in HBase from being sent to Ganglia? Or just any information on
> >> how
> >> > to configure HBase to emit fewer metrics?

Bing Jiang
Tel:(86)134-2619-1361
weibo: http://weibo.com/jiangbinglover
BLOG: www.binospace.com
BLOG: http://blog.sina.com.cn/jiangbinglover
Focus on distributed computing, HDFS/HBase
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB