Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Accumulo, mail # dev - Monitor: Adding graphs for each table and server


Copy link to this message
-
Re: Monitor: Adding graphs for each table and server
Josh Elser 2013-04-10, 15:51
Because it hasn't already been linked..
https://github.com/ericnewton/opentsdb

On 4/10/13 11:33 AM, Eric Newton wrote:
>> leverage openTSDB for Accumulo
> It was just an idea.  Good monitoring is needed for large clusters and the
> little monitor web server shows its weaknesses when the number of nodes
> gets large enough.  As a window into accumulo it is fine.  But knowing that
> disk 3 on r01n27 writes, at most, half the speed of the other 1000 drives
> is important, too.
>
> I created a tcollector plugin for accumulo so it would gather its metrics
> for TSDB.  Some accumulo useful queries or reports would probably be enough.
>
> The idea of building a better monitor around a tool more suited to
> monitoring still remains, regardless of the particular tool.
>
> -Eric
>
>
> On Wed, Apr 10, 2013 at 11:11 AM, Supun Kamburugamuva <[EMAIL PROTECTED]>wrote:
>
>> Hi Eric,
>>
>> I went through the JMX mbeans using JConsole and it seems Accumulo supports
>> a good set of metrics through JMX.
>>
>> I'm trying to figure out how to leverage openTSBD for Accumulo. From what
>> I've read so far, I haven't found an easy way to integrate with Accumulo so
>> that user can use its functionality very easily.
>>
>> Thanks,
>> Supun..
>>
>>
>> On Fri, Apr 5, 2013 at 1:34 PM, Eric Newton <[EMAIL PROTECTED]> wrote:
>>
>>> We actually have JMX support.  You need to turn it on, though.  We should
>>> just turn it on by default.
>>>
>>> Because we don't use the JMX api very much, it may need some improvement.
>>>
>>> -Eric
>>>
>>>
>>> On Fri, Apr 5, 2013 at 1:29 PM, Supun Kamburugamuva <[EMAIL PROTECTED]
>>>> wrote:
>>>> Thank you all for the input.
>>>>
>>>> I think I understand the requirement for the displaying the graphs.
>>>>
>>>> For the openTSBD part I'll go through its documentation and Eric's work
>>> to
>>>> see what I can find.
>>>>
>>>> If we are going for third party tools for monitoring, may be it is
>> better
>>>> to provide APIs (like JMX) so that users can plug in any tool.
>>>>
>>>> Thanks,
>>>> Supun..
>>>>
>>>>
>>>> On Fri, Apr 5, 2013 at 12:46 PM, Eric Newton <[EMAIL PROTECTED]>
>>>> wrote:
>>>>
>>>>>> ACCUMULO-594 <https://issues.apache.org/jira/browse/ACCUMULO-594>
>>>> talks
>>>>>> about OpenTSBD. Can you please give me some references on why we
>> need
>>>>>> OpenTSBD?
>>>>>>
>>>>> The monitor provides a quick summary of accumulo stats, a bit of
>>> history
>>>>> and the most recent ERROR/WARN log messages.
>>>>> But it's all just kept in memory and only for an hour or so.  It
>> isn't
>>> a
>>>>> real monitoring tool at all.
>>>>> It is nice to have *something* we can depend upon as soon as you get
>>>>> accumulo running.
>>>>> I do a lot of accumulo support, and having the monitor lets
>> relatively
>>>>> unsophisticated users see problems right away.
>>>>>
>>>>> But we don't have a lot of time to keep putting more energy into
>> making
>>>> the
>>>>> monitor a better monitoring tool.
>>>>> It would be better to integrate with other monitoring tools.
>>>>>
>>>>> OpenTSDB stores and presents time-series data.  It can monitor
>> anything
>>>> and
>>>>> use a NoSQL back-end database to persist the monitored data.
>>>>> It can handle monitoring thousands of nodes, each with hundreds of
>>>>> datapoints.
>>>>> Perhaps there are better tools out there.
>>>>>
>>>>> We would still need some tool for the management of log/event data,
>>> too.
>>>>> -Eric
>>>>>
>>>>
>>>>
>>>> --
>>>> Supun Kamburugamuva
>>>> Member, Apache Software Foundation; http://www.apache.org
>>>> E-mail: [EMAIL PROTECTED];  Mobile: +1 812 369 6762
>>>> Blog: http://supunk.blogspot.com
>>>>
>>
>>
>> --
>> Supun Kamburugamuva
>> Member, Apache Software Foundation; http://www.apache.org
>> E-mail: [EMAIL PROTECTED];  Mobile: +1 812 369 6762
>> Blog: http://supunk.blogspot.com
>>