Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Accumulo >> mail # user >> AgeOff Filter - Compaction needed?

Copy link to this message
Re: AgeOff Filter - Compaction needed?
Compactions re-write data. Iterators simply transform or filter data.
Configuring an iterator, such as the AgeOff iterator to a compaction
scope, means that data will be removed as compactions occur. You don't
need to force a compaction, because they happen over time, but you
might want to, to ensure the data is aged-off right away. It's
possible that an idle tablet will never compact again, and run your
age-off iterators. For that, you will need to force a compaction.

You may also want to apply your age off iterator to all scopes (minc,
majc, and scan), so that you have a consistent view in your scans
until things are permanently re-written without the old data (which
happens at the majc scope).

Christopher L Tubbs II
On Fri, Nov 22, 2013 at 10:58 AM, Slater, David M.
> Hi, I would like to automatically ageoff old data; however my understanding
> of AgeOff iterators is that they would require a compaction or scan to take
> effect. For information that is in tablets that are no longer being added to
> or scanned, the ageoff filter would not take effect, correct?
> Would I need to force a compaction or merge in order to remove those
> records, or would it be better to write my own script to remove rows? (I’m
> using 1.4 – is this behavior different in 1.5 or 1.6?)
> Thanks,
> David