Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Accumulo, mail # dev - GSOC: Monitor Improvements


+
Supun Kamburugamuva 2013-04-21, 14:37
+
Josh Elser 2013-04-21, 17:45
+
Supun Kamburugamuva 2013-04-21, 18:15
+
Josh Elser 2013-04-22, 02:00
+
Eric Newton 2013-04-22, 13:05
+
Mike Drob 2013-04-22, 15:42
+
Keith Turner 2013-04-22, 16:02
+
Supun Kamburugamuva 2013-04-22, 16:42
+
Keith Turner 2013-04-22, 17:50
+
Miguel Pereira 2013-07-24, 14:02
+
David Medinets 2013-04-22, 14:41
+
Eric Newton 2013-04-22, 14:43
Copy link to this message
-
Re: GSOC: Monitor Improvements
Josh Elser 2013-04-22, 14:57
That would be pretty sweet, actually. Potentially parallel to what you
want to do, Supun, but cool nonetheless.

I could see a lot of benefit by having some process that could emulate
the output from a non-trivially-sized Accumulo cluster on a single box.

On 4/22/13 10:43 AM, Eric Newton wrote:
> You could mock the stats collection.
>
> -Eric
>
>
> On Mon, Apr 22, 2013 at 10:41 AM, David Medinets
> <[EMAIL PROTECTED]>wrote:
>
>> The average developer probably can't access a large cluster with hundred of
>> nodes. Is there a way to simulate this?
>>
>>
>> On Mon, Apr 22, 2013 at 9:05 AM, Eric Newton <[EMAIL PROTECTED]>
>> wrote:
>>
>>> Another thing to consider is scale.  On large clusters (many hundreds of
>>> nodes), more data is not helpful for visualization.  Instead, summaries,
>>> averages and outliers are important.
>>>
>>> For example, if one node is consistently slow, it is better to know that
>>> than to see one graph with low numbers in a sea of graphs.
>>>
>>> If the monitor collects information using JMX, collection time for each
>>> node would be a good thing to know, too.
>>>
>>> -Eric
>>>
>>>
>>> On Sun, Apr 21, 2013 at 10:00 PM, Josh Elser <[EMAIL PROTECTED]>
>> wrote:
>>>> Supun,
>>>>
>>>> Yup, very much so. Having a way to consume any and all metrics via JMX
>>>> would simplify things for any consumers (internal or external).
>>>>
>>>>
>>>>
>>>> On 04/21/2013 02:15 PM, Supun Kamburugamuva wrote:
>>>>
>>>>> Hi Josh,
>>>>>
>>>>> Thanks for the suggestions. I'll incorporate these to the proposal.
>>>>>
>>>>> Another area I would like to work is on JMX. There is a Jira that says
>>> to
>>>>> replace the Monitor calls from Thrift to JMX (Accumulo 694). Do you
>>> think
>>>>> this is a good addition to the Monitor?
>>>>>
>>>>> Thanks,
>>>>> Supun..
>>>>>
>>>>>
>>>>> On Sun, Apr 21, 2013 at 1:45 PM, Josh Elser <[EMAIL PROTECTED]>
>>> wrote:
>>>>>   Supun,
>>>>>> Looks good! Can I make some suggestions/comments?
>>>>>>
>>>>>> For: "Per table plots: ACCUMULO-594", I'd also like to see minor
>>>>>> compactions, major compactions, index cache hit rate, and data cache
>>> hit
>>>>>> rate per table (same graphs that are displayed system-wide when you
>>> visit
>>>>>> http://${MONITOR_HOST}:50095/.
>>>>>>
>>>>>> For "Per tablet [server] plots", it would be neat if you could also
>>>>>> extract some general statistics like top N least performing, top N
>>>>>> highest
>>>>>> performing, etc. tablet servers. Ideally, this could correlate with
>>>>>> servers
>>>>>> that may be having problems :).
>>>>>>
>>>>>> Do you see these proposed changes as being sufficient for 3-4 months
>> of
>>>>>> 40hrs/week work? If you plan to really dig into these changes
>> (perhaps
>>>>>> reworking components of the monitor itself), I could perhaps see
>> this.
>>> Do
>>>>>> you have any ideas for more lofty goals that you could pursue as
>> well?
>>> I
>>>>>> don't want you/us to get one month into things and see you complete
>>>>>> everything we initially planned to accomplish :)
>>>>>>
>>>>>> - Josh
>>>>>>
>>>>>>
>>>>>> On 04/21/2013 10:37 AM, Supun Kamburugamuva wrote:
>>>>>>
>>>>>>   Hi all,
>>>>>>> I would like to start writing the proposal for the GSoc. I've put
>>>>>>> together
>>>>>>> some initial high level goals of the project. Please let me know
>> what
>>> I
>>>>>>> can
>>>>>>> improve.
>>>>>>>
>>>>>>> Per table plots: Accumulo 594
>>>>>>> ---------------------
>>>>>>>
>>>>>>> The goal of this is to display plots that explains the various
>>>>>>> activtities
>>>>>>> that happens per table. When we go to the tables page of the monitor
>>> and
>>>>>>> go
>>>>>>> to a specific table it displays some information in a table format.
>> We
>>>>>>> can
>>>>>>> argument this information by showing graphs for
>>>>>>>
>>>>>>> 1. Ingest entries
>>>>>>> 2. Ingest data size
>>>>>>> 3. Scan entries
>>>>>>> 4. Scan data size
>>>>>>>
>>>>>>> Per tablet plots
>>>>>>> ----------------------
>>>>>>>
>>>>>>> Same as in the table plots we can display information regarding
+
Supun Kamburugamuva 2013-04-22, 15:04
+
Eric Newton 2013-04-22, 15:17
+
Supun Kamburugamuva 2013-04-22, 15:27
+
Supun Kamburugamuva 2013-04-22, 14:33
+
Eric Newton 2013-04-22, 15:03
+
Gabe Bell 2013-04-22, 15:09