Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Accumulo, mail # dev - metric gathering


Copy link to this message
-
Re: metric gathering
Miguel Pereira 2012-07-16, 20:44
Thanks for the help Eric,

I successfully queried accumulo metrics with jmxtrans and passed them to
ganglia.
Going to try it with an actual cluster before posting instructions..

Cheers,
Miguel

On Fri, Jul 13, 2012 at 4:07 PM, Eric Newton <[EMAIL PROTECTED]> wrote:

> You can put them in your accumulo-env.sh.  Be sure to enable the
> metrics in the accumulo-metrix.xml file, too.
>
> I did this when I ported OpenTSDB to accumulo.  It worked great.
>
> -Eric
>
> On Fri, Jul 13, 2012 at 3:44 PM, Miguel Pereira
> <[EMAIL PROTECTED]> wrote:
> > Hey guys,
> >
> > were would I enable a jmx port for accumulo?
> > the -D options i want to use are
> >
> > -Dcom.sun.management.jmxremote.port=1105
> > -Dcom.sun.management.jmxremote.authenticate=false
> > -Dcom.sun.management.jmxremote.ssl=false
> >
> > but i don't know were to put them, accumulo-env.sh ?
> > I plan to use jmxtrans to feed metrics to ganglia & maybe graphite
> >
> > Cheers,
> > Miguel
> >
> > On Fri, Jun 29, 2012 at 6:52 PM, Eric Newton <[EMAIL PROTECTED]>
> wrote:
> >
> >> Edit accumulo-metrics.xml, turn the false settings to true, restart,
> >> and then look at the exposed JMX beans with jconsole. You may need to
> >> expose a jmx port with various "-D" options, none of which I remember
> >> how to do.
> >>
> >> I don't know much about ganglia.  I've used it a little, but
> >> displaying 100's of graphs on a web page is almost as bad as the the
> >> lack of visualization and historical data in the accumulo monitor.  I
> >> don't know how to configure it.
> >>
> >> I didn't find it difficult to modify tcollector, the performance
> >> collector for OpenTSDB, to collect metrics for the Accumulo tservers.
> >>
> >> If you do manage it, please post instructions!
> >>
> >> -Eric
> >>
> >> On Fri, Jun 29, 2012 at 5:07 PM, Miguel Pereira
> >> <[EMAIL PROTECTED]> wrote:
> >> > I was hoping the accumulo conf dir would have a
> hadoop-metrics.properties
> >> > file similar to hbase were you just drop these lines in..
> >> >
> >> > hbase.class=org.apache.hadoop.metrics.ganglia.GangliaContext
> >> > hbase.period=10
> >> > hbase.servers=master.yourcompany.com:8649
> >> >
> >> > jvm.class=org.apache.hadoop.metrics.ganglia.GangliaContext
> >> > jvm.period=10
> >> > jvm.servers=master.yourcompany.com:8649
> >> >
> >> > rpc.class=org.apache.hadoop.metrics.ganglia.GangliaContext
> >> > rpc.period=10
> >> > rpc.servers=master.yourcompany.com:8649
> >> >
> >> >
> >> > I did spot the accumulo-metrics.xml and the metrics page in the doc
> >> > directory. Anyone have a few examples of how to use JMX here, I feel
> >> like i
> >> > am really close.
> >> >
> >> > cheers and have a great weekend,
> >> > Miguel
> >> >
> >> >
> >> > On Fri, Jun 29, 2012 at 2:00 PM, Eric Newton <[EMAIL PROTECTED]>
> >> wrote:
> >> >
> >> >> I do most of my monitoring (and first-line trouble-shooting) via the
> >> >> Accumulo monitor page.
> >> >>
> >> >> I know most teams use XML to import data into their monitoring
> systems
> >> >> (based on the complaints I received when I changed the XML).
> >> >>
> >> >> I did port OpenTSDB to Accumulo; and it reads data via JMX.
> >> >>
> >> >> I used to work for Zenoss, so I'm partial to it, too.
> >> >>
> >> >> -Eric
> >> >>
> >> >> On Fri, Jun 29, 2012 at 11:55 AM, Miguel Pereira
> >> >> <[EMAIL PROTECTED]> wrote:
> >> >> > Thanks Eric!
> >> >> >
> >> >> > Are you guys currently using ganglia or is most of your monitoring
> >> done
> >> >> > through jmx?
> >> >> >
> >> >> > -Miguel
> >> >> >
> >> >> > On Fri, Jun 29, 2012 at 11:09 AM, Eric Newton <
> [EMAIL PROTECTED]>
> >> >> wrote:
> >> >> >
> >> >> >> You have three options:
> >> >> >>
> >> >> >> 1) You can turn on JMX monitoring.
> >> >> >> 2) You can use org.apache.accumulo.server.test.GetMasterStats to
> >> print
> >> >> >> the data the monitor uses
> >> >> >> 3) You can pull the xml from the monitor page
> >> >> >>
> >> >> >> -Eric
> >> >> >>
> >> >> >> On Fri, Jun 29, 2012 at 11:03 AM, Miguel Pereira