Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> quieting HBase metrics


Copy link to this message
-
Re: quieting HBase metrics
hi, Ron.
Indeed, I have rewrote the implementation of
org.apache.hadoop.hbase.regionserver.metrics.RegionServerMetrics
org.apache.hadoop.hbase.regionserver.metrics.RegionServerDynamicMetrics
There are major changes as below:
1) change the context name for hbase to separate name for each other;
like hadoop-metrics.properties:
hbase-rsdm respresents DynamicMetrics
and hbase-rs represents RegionServerMetrics.

2) add a filterMapper in Metrics, when it updates to ganglia or other
metrics collector.
so in our implementation, it will fetch the filter from
hadoop-metrics.properties, and emit metrics as to settings.

3) In order to support dynamic add/remove filter in
hadoop-metrics.properties, we alternatively add a rpc to restart metrics
system in RegionServer without restarting rs.

Tks
2013/9/24 Ron Echeverri <[EMAIL PROTECTED]>

> Thanks, Bing, but i still don't understand *how* to configure that.  Could
> you share your hadoop-metrics.properties file, or any other files you use
> for this?
>
> rone
>
>
> On Mon, Sep 23, 2013 at 8:29 PM, Bing Jiang <[EMAIL PROTECTED]
> >wrote:
>
> > hbase metrics contains different types:RegionServerMetrics and
> > RegionServerDynamicMetrics.
> > And there are more metrics emitted from RegionServerDynamicMetrics,
> > especially in case that there are more regions in RS.
> > RegionServerMetrics and RSDM share one metrics context, which makes the
> > control become difficult.
> >
> > In our production, we add a metrics filter in MetricsContext, which can
> be
> > configured as to our needs, which will reduce the amount of metrics, and
> > take a good effect.
> >
> >
> >
> >
> > 2013/9/24 Otis Gospodnetic <[EMAIL PROTECTED]>
> >
> >> Me, too!
> >>
> >> Otis
> >> --
> >> HBase Performance Monitoring -- http://sematext.com/spm
> >>
> >>
> >>
> >> On Mon, Sep 23, 2013 at 7:10 AM, Arati Patro <[EMAIL PROTECTED]>
> >> wrote:
> >> > Hi All,
> >> >
> >> >
> >> > I have the same query. Tried searching for a solution on the web but
> >> found
> >> > nothing very helpful. Is there some way I could disable metrics for
> >> every
> >> > table in HBase from being sent to Ganglia? Or just any information on
> >> how
> >> > to configure HBase to emit fewer metrics?
> >> >
> >> >
> >> > Thanks,
> >> > Arati Patro
> >> >
> >> >
> >> > On Thu, Sep 19, 2013 at 4:47 AM, Ron Echeverri <
> [EMAIL PROTECTED]
> >> >wrote:
> >> >
> >> >> In http://hbase.apache.org/book/hbase_metrics.html we see
> >> >>
> >> >> """
> >> >> 15.4.2. Warning To Ganglia Users
> >> >>
> >> >> Warning to Ganglia Users: by default, HBase will emit a LOT of
> metrics
> >> per
> >> >> RegionServer which may swamp your installation. Options include
> either
> >> >> increasing Ganglia server capacity, or configuring HBase to emit
> fewer
> >> >> metrics.
> >> >> """
> >> >>
> >> >> However, although there is documentation to help set up Ganglia
> >> metrics for
> >> >> HBase, there doesn't seem to be any that shows how to make HBase emit
> >> fewer
> >> >> metrics.  Could someone lend me a hand?  I have increased the period
> in
> >> >> hadoop-metrics.properties but that has proven insufficient.
> >> >>
> >> >> thanks
> >> >> rone
> >> >>
> >>
> >
> >
> >
> > --
> > Bing Jiang
> > weibo: http://weibo.com/jiangbinglover
> > BLOG: www.binospace.com
> > BLOG: http://blog.sina.com.cn/jiangbinglover
> > Focus on distributed computing, HDFS/HBase
> >
> >
> > 2013/9/24 Otis Gospodnetic <[EMAIL PROTECTED]>
> >
> >> Me, too!
> >>
> >> Otis
> >> --
> >> HBase Performance Monitoring -- http://sematext.com/spm
> >>
> >>
> >>
> >> On Mon, Sep 23, 2013 at 7:10 AM, Arati Patro <[EMAIL PROTECTED]>
> >> wrote:
> >> > Hi All,
> >> >
> >> >
> >> > I have the same query. Tried searching for a solution on the web but
> >> found
> >> > nothing very helpful. Is there some way I could disable metrics for
> >> every
> >> > table in HBase from being sent to Ganglia? Or just any information on
> >> how
> >> > to configure HBase to emit fewer metrics?

Bing Jiang
Tel:(86)134-2619-1361
weibo: http://weibo.com/jiangbinglover
BLOG: www.binospace.com
BLOG: http://blog.sina.com.cn/jiangbinglover
Focus on distributed computing, HDFS/HBase