Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Accumulo, mail # dev - GSOC: Monitor Improvements


+
Supun Kamburugamuva 2013-04-21, 14:37
+
Josh Elser 2013-04-21, 17:45
+
Supun Kamburugamuva 2013-04-21, 18:15
+
Josh Elser 2013-04-22, 02:00
+
Eric Newton 2013-04-22, 13:05
+
Mike Drob 2013-04-22, 15:42
+
Keith Turner 2013-04-22, 16:02
+
Supun Kamburugamuva 2013-04-22, 16:42
+
Keith Turner 2013-04-22, 17:50
+
Miguel Pereira 2013-07-24, 14:02
+
David Medinets 2013-04-22, 14:41
Copy link to this message
-
Re: GSOC: Monitor Improvements
Eric Newton 2013-04-22, 14:43
You could mock the stats collection.

-Eric
On Mon, Apr 22, 2013 at 10:41 AM, David Medinets
<[EMAIL PROTECTED]>wrote:

> The average developer probably can't access a large cluster with hundred of
> nodes. Is there a way to simulate this?
>
>
> On Mon, Apr 22, 2013 at 9:05 AM, Eric Newton <[EMAIL PROTECTED]>
> wrote:
>
> > Another thing to consider is scale.  On large clusters (many hundreds of
> > nodes), more data is not helpful for visualization.  Instead, summaries,
> > averages and outliers are important.
> >
> > For example, if one node is consistently slow, it is better to know that
> > than to see one graph with low numbers in a sea of graphs.
> >
> > If the monitor collects information using JMX, collection time for each
> > node would be a good thing to know, too.
> >
> > -Eric
> >
> >
> > On Sun, Apr 21, 2013 at 10:00 PM, Josh Elser <[EMAIL PROTECTED]>
> wrote:
> >
> > > Supun,
> > >
> > > Yup, very much so. Having a way to consume any and all metrics via JMX
> > > would simplify things for any consumers (internal or external).
> > >
> > >
> > >
> > > On 04/21/2013 02:15 PM, Supun Kamburugamuva wrote:
> > >
> > >> Hi Josh,
> > >>
> > >> Thanks for the suggestions. I'll incorporate these to the proposal.
> > >>
> > >> Another area I would like to work is on JMX. There is a Jira that says
> > to
> > >> replace the Monitor calls from Thrift to JMX (Accumulo 694). Do you
> > think
> > >> this is a good addition to the Monitor?
> > >>
> > >> Thanks,
> > >> Supun..
> > >>
> > >>
> > >> On Sun, Apr 21, 2013 at 1:45 PM, Josh Elser <[EMAIL PROTECTED]>
> > wrote:
> > >>
> > >>  Supun,
> > >>>
> > >>> Looks good! Can I make some suggestions/comments?
> > >>>
> > >>> For: "Per table plots: ACCUMULO-594", I'd also like to see minor
> > >>> compactions, major compactions, index cache hit rate, and data cache
> > hit
> > >>> rate per table (same graphs that are displayed system-wide when you
> > visit
> > >>> http://${MONITOR_HOST}:50095/.
> > >>>
> > >>> For "Per tablet [server] plots", it would be neat if you could also
> > >>> extract some general statistics like top N least performing, top N
> > >>> highest
> > >>> performing, etc. tablet servers. Ideally, this could correlate with
> > >>> servers
> > >>> that may be having problems :).
> > >>>
> > >>> Do you see these proposed changes as being sufficient for 3-4 months
> of
> > >>> 40hrs/week work? If you plan to really dig into these changes
> (perhaps
> > >>> reworking components of the monitor itself), I could perhaps see
> this.
> > Do
> > >>> you have any ideas for more lofty goals that you could pursue as
> well?
> > I
> > >>> don't want you/us to get one month into things and see you complete
> > >>> everything we initially planned to accomplish :)
> > >>>
> > >>> - Josh
> > >>>
> > >>>
> > >>> On 04/21/2013 10:37 AM, Supun Kamburugamuva wrote:
> > >>>
> > >>>  Hi all,
> > >>>>
> > >>>> I would like to start writing the proposal for the GSoc. I've put
> > >>>> together
> > >>>> some initial high level goals of the project. Please let me know
> what
> > I
> > >>>> can
> > >>>> improve.
> > >>>>
> > >>>> Per table plots: Accumulo 594
> > >>>> ---------------------
> > >>>>
> > >>>> The goal of this is to display plots that explains the various
> > >>>> activtities
> > >>>> that happens per table. When we go to the tables page of the monitor
> > and
> > >>>> go
> > >>>> to a specific table it displays some information in a table format.
> We
> > >>>> can
> > >>>> argument this information by showing graphs for
> > >>>>
> > >>>> 1. Ingest entries
> > >>>> 2. Ingest data size
> > >>>> 3. Scan entries
> > >>>> 4. Scan data size
> > >>>>
> > >>>> Per tablet plots
> > >>>> ----------------------
> > >>>>
> > >>>> Same as in the table plots we can display information regarding
> tablet
> > >>>> servers in the tablet server page. The plots will display the same
> > >>>> information as table plots considering data per tablet server.
> > >>>>
> > >>>> Trace Visualization: Accumulo 1198
+
Josh Elser 2013-04-22, 14:57
+
Supun Kamburugamuva 2013-04-22, 15:04
+
Eric Newton 2013-04-22, 15:17
+
Supun Kamburugamuva 2013-04-22, 15:27
+
Supun Kamburugamuva 2013-04-22, 14:33
+
Eric Newton 2013-04-22, 15:03
+
Gabe Bell 2013-04-22, 15:09