Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Accumulo >> mail # user >> AgeOff Filter - Compaction needed?


Copy link to this message
-
Re: AgeOff Filter - Compaction needed?
Compactions re-write data. Iterators simply transform or filter data.
Configuring an iterator, such as the AgeOff iterator to a compaction
scope, means that data will be removed as compactions occur. You don't
need to force a compaction, because they happen over time, but you
might want to, to ensure the data is aged-off right away. It's
possible that an idle tablet will never compact again, and run your
age-off iterators. For that, you will need to force a compaction.

You may also want to apply your age off iterator to all scopes (minc,
majc, and scan), so that you have a consistent view in your scans
until things are permanently re-written without the old data (which
happens at the majc scope).

--
Christopher L Tubbs II
http://gravatar.com/ctubbsii
On Fri, Nov 22, 2013 at 10:58 AM, Slater, David M.
<[EMAIL PROTECTED]> wrote:
> Hi, I would like to automatically ageoff old data; however my understanding
> of AgeOff iterators is that they would require a compaction or scan to take
> effect. For information that is in tablets that are no longer being added to
> or scanned, the ageoff filter would not take effect, correct?
>
>
>
> Would I need to force a compaction or merge in order to remove those
> records, or would it be better to write my own script to remove rows? (I’m
> using 1.4 – is this behavior different in 1.5 or 1.6?)
>
>
>
> Thanks,
> David
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB