Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Accumulo >> mail # dev >> Monitor: Adding graphs for each table and server


Copy link to this message
-
Re: Monitor: Adding graphs for each table and server
> leverage openTSDB for Accumulo

It was just an idea.  Good monitoring is needed for large clusters and the
little monitor web server shows its weaknesses when the number of nodes
gets large enough.  As a window into accumulo it is fine.  But knowing that
disk 3 on r01n27 writes, at most, half the speed of the other 1000 drives
is important, too.

I created a tcollector plugin for accumulo so it would gather its metrics
for TSDB.  Some accumulo useful queries or reports would probably be enough.

The idea of building a better monitor around a tool more suited to
monitoring still remains, regardless of the particular tool.

-Eric
On Wed, Apr 10, 2013 at 11:11 AM, Supun Kamburugamuva <[EMAIL PROTECTED]>wrote:

> Hi Eric,
>
> I went through the JMX mbeans using JConsole and it seems Accumulo supports
> a good set of metrics through JMX.
>
> I'm trying to figure out how to leverage openTSBD for Accumulo. From what
> I've read so far, I haven't found an easy way to integrate with Accumulo so
> that user can use its functionality very easily.
>
> Thanks,
> Supun..
>
>
> On Fri, Apr 5, 2013 at 1:34 PM, Eric Newton <[EMAIL PROTECTED]> wrote:
>
> > We actually have JMX support.  You need to turn it on, though.  We should
> > just turn it on by default.
> >
> > Because we don't use the JMX api very much, it may need some improvement.
> >
> > -Eric
> >
> >
> > On Fri, Apr 5, 2013 at 1:29 PM, Supun Kamburugamuva <[EMAIL PROTECTED]
> > >wrote:
> >
> > > Thank you all for the input.
> > >
> > > I think I understand the requirement for the displaying the graphs.
> > >
> > > For the openTSBD part I'll go through its documentation and Eric's work
> > to
> > > see what I can find.
> > >
> > > If we are going for third party tools for monitoring, may be it is
> better
> > > to provide APIs (like JMX) so that users can plug in any tool.
> > >
> > > Thanks,
> > > Supun..
> > >
> > >
> > > On Fri, Apr 5, 2013 at 12:46 PM, Eric Newton <[EMAIL PROTECTED]>
> > > wrote:
> > >
> > > > > ACCUMULO-594 <https://issues.apache.org/jira/browse/ACCUMULO-594>
> > > talks
> > > > > about OpenTSBD. Can you please give me some references on why we
> need
> > > > > OpenTSBD?
> > > > >
> > > >
> > > > The monitor provides a quick summary of accumulo stats, a bit of
> > history
> > > > and the most recent ERROR/WARN log messages.
> > > > But it's all just kept in memory and only for an hour or so.  It
> isn't
> > a
> > > > real monitoring tool at all.
> > > > It is nice to have *something* we can depend upon as soon as you get
> > > > accumulo running.
> > > > I do a lot of accumulo support, and having the monitor lets
> relatively
> > > > unsophisticated users see problems right away.
> > > >
> > > > But we don't have a lot of time to keep putting more energy into
> making
> > > the
> > > > monitor a better monitoring tool.
> > > > It would be better to integrate with other monitoring tools.
> > > >
> > > > OpenTSDB stores and presents time-series data.  It can monitor
> anything
> > > and
> > > > use a NoSQL back-end database to persist the monitored data.
> > > > It can handle monitoring thousands of nodes, each with hundreds of
> > > > datapoints.
> > > > Perhaps there are better tools out there.
> > > >
> > > > We would still need some tool for the management of log/event data,
> > too.
> > > >
> > > > -Eric
> > > >
> > >
> > >
> > >
> > > --
> > > Supun Kamburugamuva
> > > Member, Apache Software Foundation; http://www.apache.org
> > > E-mail: [EMAIL PROTECTED];  Mobile: +1 812 369 6762
> > > Blog: http://supunk.blogspot.com
> > >
> >
>
>
>
> --
> Supun Kamburugamuva
> Member, Apache Software Foundation; http://www.apache.org
> E-mail: [EMAIL PROTECTED];  Mobile: +1 812 369 6762
> Blog: http://supunk.blogspot.com
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB