Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> Is it feasible to delete qualified tombstones during minor compaction?


Copy link to this message
-
Re: Is it feasible to delete qualified tombstones during minor compaction?
Actually, major compaction still doesn't look at memstore so the window for
reappearing puts is still there, it's just small. I am +1 on the OP idea :)

On Thu, Mar 14, 2013 at 10:00 AM, lars hofhansl <[EMAIL PROTECTED]> wrote:

> Unless you can guarantee that all KVs affected by a delete marker have
> been deleted you cannot remove the delete marker. That's a fact :)
> That means in order to remove a delete marker you have to look everywhere
> where a potentially affected KV could reside. Currently that is everywhere
> and hence only a major compaction can do that.
>
> Sergey is working on striped compactions where will be able (hopefully) to
> limit where we have to look for affected to something smaller than
> "everything".
>
> -- Lars
>
>
>
> ________________________________
>  From: Ted Yu <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]
> Sent: Thursday, March 14, 2013 9:41 AM
> Subject: Re: Is it feasible to delete qualified tombstones during minor
> compaction?
>
> That is correct.
>
> On Thu, Mar 14, 2013 at 9:28 AM, ramkrishna vasudevan <
> [EMAIL PROTECTED]> wrote:
>
> > So seeing the final patch it means that whatever Liang has asked for is
> not
> > available right?
> >
> > Regards
> > Ram
> >
> > On Thu, Mar 14, 2013 at 9:55 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
> >
> > > The issue was HBASE-7902
> > >
> > > The final patch was just mostly renaming:
> > >
> > > -          Collections.singletonList(memstoreScanner),
> > > ScanType.MINOR_COMPACT,
> > > +          Collections.singletonList(memstoreScanner),
> > > ScanType.COMPACT_RETAIN_DELETES,
> > >
> > > Cheers
> > >
> > > On Thu, Mar 14, 2013 at 9:22 AM, Anoop John <[EMAIL PROTECTED]>
> > wrote:
> > >
> > > > Hi
> > > >          I have seen some issue raised by Sergey regarding removing
> > > Delete
> > > > markers during minor compaction. Not remembering the issue id and
> > details
> > > > :(
> > > >
> > > > -Anoop-
> > > >
> > > >
> > > > On Thu, Mar 14, 2013 at 3:11 PM, 谢良 <[EMAIL PROTECTED]> wrote:
> > > >
> > > > > Currently, all tombstones only get cleaned up with major
> > compactions. I
> > > > > guess the original
> > > > > design probably considered that we need to ensure whether there are
> > > other
> > > > > versions of
> > > > > the deleted key in other storefiles which are not included in minor
> > > > > compaction? If that's
> > > > > true, i have a thought, maybe we could delete a bunch of tombstones
> > per
> > > > > bloom filter, in
> > > > > other words, we can try to query the deleted key in other
> > storefiles's
> > > > > bloom filter info to determine
> > > > > whether we can remove it safely or not.
> > > > > I am not familiar with compaction&bloomfilter internal, it's just a
> > raw
> > > > > thought right now:)
> > > > >
> > > > > Regards,
> > > > > Liang
> > > >
> > >
> >
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB