Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - Extending HBase Metrics Framework


Copy link to this message
-
Re: Extending HBase Metrics Framework
Joarder KAMAL 2013-06-06, 18:53
Thanks for pointing out the way. I've looked at the Git code as well
however looking back at the apidocs <http://hbase.apache.org/apidocs/> for
HBase-0.97.0-Snapshot I can only see org.apache.hadoop.metrics2 package
there.

I am using HBase 0.94.7, could you kindly help directing to the right code
repo and package?

Thanks again...
Regards,
Joarder Kamal
On 7 June 2013 03:50, Jeff Kolesky <[EMAIL PROTECTED]> wrote:

> If you want to add metrics, you can implement a
> org.apache.hadoop.metrics.Updater and register it with a MetricsContext.
>  There is an example at:
>
>
> https://github.com/jeffkole/odoop/blob/master/hbase-filters/src/main/java/com/opower/hadoop/hbase/filter/DeployedFilterMetrics.java
>
> This exposes metrics for a filter that I wrote, but if you want to pull out
> metrics for HBase internals, you likely need to do modify HBase source code
> to get at the data or hook up to your metric updater.
>
> Jeff
>
>
> On Thu, Jun 6, 2013 at 10:35 AM, Joarder KAMAL <[EMAIL PROTECTED]> wrote:
>
> > Hi Bing,
> >
> > Thanks for answering.
> >
> > 1. Probably my question was unclear. I was asking whether it is possible
> to
> > generate custom metrics from HBase by modifying the source code.
> > 2. Thanks for letting me know that it is possible to log the metrics in
> any
> > kind of file. But I guess it would be still in XML format !! Can I save
> the
> > logs in raw text format?? I'll try to do this tomorrow.
> >
> > Thanks again.
> >
> >
> > Regards,
> > Joarder Kamal
> >
> >
> >
> > On 6 June 2013 17:42, Bing Jiang <[EMAIL PROTECTED]> wrote:
> >
> > > 1) Metrics infos have been collected from HMaster\RS, and covering rpc,
> > > jvm, hbase functional module (like compaction\flush\request
> > Count\DataBlock
> > > read &&write etc).
> > >
> > > 2) You can set metrics info handler in conf/hadoop-metrics.properties
> > > for example, you can set hbase.class > > > org.apache.hadoop.metrics.ganglia.GangliaContext31, it will tell
> Metrics
> > > System where the metrics will be sent to. Of course, this is Ganglia.
> > > if set
> > >
> hbase.class=org.apache.hadoop.hbase.metrics.file.TimeStampingFileContext,
> > > it will write the metrics to file.
> > >
> > > # hbase.class=org.apache.hadoop.metrics.ganglia.GangliaContext
> > > # hbase.class=org.apache.hadoop.metrics.ganglia.GangliaContext31
> > > # hbase.period=10
> > > # hbase.servers=10.100.4.151:8650
> > >
> #hbase.class=org.apache.hadoop.hbase.metrics.file.TimeStampingFileContext
> > > #hbase.period=10
> > > #hbase.fileName=/opt/hbase/logs/metrics_hbase.log
> > >
> > >
> > >
> > >
> > > 2013/6/6 Joarder KAMAL <[EMAIL PROTECTED]>
> > >
> > > > Two Questions:
> > > >
> > > > 1. Is it possible to extend HBase metrics/reporting framework to
> > generate
> > > > custom metrics? I was looking at the api docs
> > > > http://hbase.apache.org/apidocs/
> > > > 2. Is there any way to write my own client (instead of using Ganglia
> or
> > > > else) to retrieve the currently available metrics?
> > > >
> > > > HBase version: 0.94.7
> > > >
> > > > Any suggestions will be very helpful.
> > > >
> > > > Many thanks ...
> > > >
> > > > Regards,
> > > > Joarder Kamal
> > > >
> > >
> > >
> > >
> > > --
> > > Bing Jiang
> > > Tel:(86)134-2619-1361
> > > weibo: http://weibo.com/jiangbinglover
> > > BLOG: http://blog.sina.com.cn/jiangbinglover
> > > National Research Center for Intelligent Computing Systems
> > > Institute of Computing technology
> > > Graduate University of Chinese Academy of Science
> > >
> >
>
>
>
> --
> *Jeff Kolesky*
> Chief Software Architect
> *Opower*
>