Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Accumulo >> mail # user >> OpenTSDB and Accumulo


+
Eric Newton 2012-10-03, 21:29
+
Ed Kohlwey 2012-10-04, 16:44
Copy link to this message
-
Re: OpenTSDB and Accumulo
I don't know, but it should be fairly easy; there's a REST API.  You can
always pull datapoints out of accumulo yourself.  I wish I was better at
visualization.

Sometimes, when doing post-failure analysis, it would be *really* nice to
know:

"Hey, was accumulo pushed into swap?"
"Did real memory get low?"
"Was java GC taking more and more time?"
"Huh... the New Generation was really getting huge."

So, that's my interest in it: gathering additional information, so after a
partial failure, we can determine what should be done to prevent problems
in the future.

At this point, it's good enough to use the built-in web server to explore
the datapoints during a failure event.

One thing that would be nice: analyse I/O performance over time to find
hardware that is having problems, but has not yet failed.

-Eric

On Thu, Oct 4, 2012 at 12:44 PM, Ed Kohlwey <[EMAIL PROTECTED]> wrote:

> Eric,
> This is a very cool project. I've heard of OpenTSDB before but hadn't ever
> had the chance to fiddle with it.
>
> Do you know what visualization and analytics packages are available on top
> of it?
>
>
> On Wed, Oct 3, 2012 at 5:29 PM, Eric Newton <[EMAIL PROTECTED]> wrote:
>
>> A while back, I ported OpenTSDB to Accumulo.  I set it up on a real
>> (test) cluster, and let it collect some data points.
>>
>> Eyecandy screenshot attached.  I was just playing around with it.  The
>> graph shows the culmulative load vs. the queued minor compactions.
>>  Probably not very useful, but it made a nice picture.
>>
>> OpenTSDB: http://opentsdb.net
>> OpenTSDB over Accumulo: https://github.com/ericnewton/opentsdb
>>
>> Accumulo can be monitored with the standard OpenTSDB, too. You need to
>> turn on the JMX metrics (conf/accumulo-metrics.xml).  I created a script to
>> collect tablet server metrics in my copy of tcollector (
>> https://github.com/ericnewton/ <https://github.com/ericnewton/opentsdb>
>> tcollector).
>>
>> -Eric
>>
>>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB