-Re: Many locality groups
Josh Elser 2013-09-18, 16:01
Glad to see I wasn't completely off base with some of the complexity
numbers I was expecting. I'll pick up my poking and prodding where you left
On Wed, Sep 18, 2013 at 11:35 AM, Keith Turner <[EMAIL PROTECTED]> wrote:
> I ran some test before and after partitioning tablet memory in
> ACCUMULO-112. I commented on the performance numbers I saw. I checked in
> the code I used to test.
> Looking back at the test, one thing I did not time was reading all of the
> locality groups in scan.
> On Wed, Sep 18, 2013 at 11:02 AM, Josh Elser <[EMAIL PROTECTED]> wrote:
> > I have a use case in which I'm investigating setting a locality group on
> > every column family in a table which has very "dense" rows (many columns
> > appear within the same tablet).
> > When scanning over a single column, I see a slow-down as one might expect
> > (filtering out the columns I don't care about). Setting each column into
> > its own locality group helps speed things up again for that single column
> > query case.
> > I'm curious if anyone has any insight to when/if I'm going to start
> > a penalty for having many locality groups. Glancing back over
> > I have to read each LocalityGroupMetadata and its multi-level index
> > shouldn't be bad if I remember Keith's talks) and then I should get
> > reads across the locality groups I need to open.
> > Is the same true for writing data to many a table with many locality
> > groups? Nothing terrible pops out at me looking at the code.
> > I was planning to write some tests to try and simulate this, but figured
> > can poll the community as well to see if anyone has experimented in this
> > scenario before.
> > Thanks!
> > - Josh