Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Accumulo, mail # user - Running Accumulo straight from Memory


Copy link to this message
-
Re: Running Accumulo straight from Memory
Keith Turner 2012-09-13, 16:32
On Wed, Sep 12, 2012 at 5:20 PM, David Medinets
<[EMAIL PROTECTED]> wrote:
> Why would locality groups be useful in an in-memory system?

Memory is fast, yet we still organize data in memory to make it really
fast (e.g. hash maps, sorted maps, bloom filters, etc)   Locality
groups are no different.  If using that data organization will make
what you are attempting to do faster, then you would probably use it.
Assume you have two locality groups and one contains 1% of your data
by volume and the other 99%.    Scanning just the locality group with
1% of the data will be faster than not having locality groups.  It
cuts down on the amount of data you have to read and processes from
memory.

>
> On Wed, Sep 12, 2012 at 4:53 PM, Adam Fuchs <[EMAIL PROTECTED]> wrote:
>> Even if you are just using memory, minor and major compactions are important
>> to get compression, handle deletes, get sequential access (cache line
>> efficiency), use iterators, and introduce locality groups.