Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # dev >> Is it feasible to delete qualified tombstones during minor compaction?


+
谢良 2013-03-14, 09:41
+
ramkrishna vasudevan 2013-03-14, 13:51
+
Anoop John 2013-03-14, 16:22
+
Ted Yu 2013-03-14, 16:25
+
ramkrishna vasudevan 2013-03-14, 16:28
+
Ted Yu 2013-03-14, 16:41
+
lars hofhansl 2013-03-14, 17:00
Copy link to this message
-
Re: Is it feasible to delete qualified tombstones during minor compaction?
Actually, major compaction still doesn't look at memstore so the window for
reappearing puts is still there, it's just small. I am +1 on the OP idea :)

On Thu, Mar 14, 2013 at 10:00 AM, lars hofhansl <[EMAIL PROTECTED]> wrote:

> Unless you can guarantee that all KVs affected by a delete marker have
> been deleted you cannot remove the delete marker. That's a fact :)
> That means in order to remove a delete marker you have to look everywhere
> where a potentially affected KV could reside. Currently that is everywhere
> and hence only a major compaction can do that.
>
> Sergey is working on striped compactions where will be able (hopefully) to
> limit where we have to look for affected to something smaller than
> "everything".
>
> -- Lars
>
>
>
> ________________________________
>  From: Ted Yu <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]
> Sent: Thursday, March 14, 2013 9:41 AM
> Subject: Re: Is it feasible to delete qualified tombstones during minor
> compaction?
>
> That is correct.
>
> On Thu, Mar 14, 2013 at 9:28 AM, ramkrishna vasudevan <
> [EMAIL PROTECTED]> wrote:
>
> > So seeing the final patch it means that whatever Liang has asked for is
> not
> > available right?
> >
> > Regards
> > Ram
> >
> > On Thu, Mar 14, 2013 at 9:55 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
> >
> > > The issue was HBASE-7902
> > >
> > > The final patch was just mostly renaming:
> > >
> > > -          Collections.singletonList(memstoreScanner),
> > > ScanType.MINOR_COMPACT,
> > > +          Collections.singletonList(memstoreScanner),
> > > ScanType.COMPACT_RETAIN_DELETES,
> > >
> > > Cheers
> > >
> > > On Thu, Mar 14, 2013 at 9:22 AM, Anoop John <[EMAIL PROTECTED]>
> > wrote:
> > >
> > > > Hi
> > > >          I have seen some issue raised by Sergey regarding removing
> > > Delete
> > > > markers during minor compaction. Not remembering the issue id and
> > details
> > > > :(
> > > >
> > > > -Anoop-
> > > >
> > > >
> > > > On Thu, Mar 14, 2013 at 3:11 PM, 谢良 <[EMAIL PROTECTED]> wrote:
> > > >
> > > > > Currently, all tombstones only get cleaned up with major
> > compactions. I
> > > > > guess the original
> > > > > design probably considered that we need to ensure whether there are
> > > other
> > > > > versions of
> > > > > the deleted key in other storefiles which are not included in minor
> > > > > compaction? If that's
> > > > > true, i have a thought, maybe we could delete a bunch of tombstones
> > per
> > > > > bloom filter, in
> > > > > other words, we can try to query the deleted key in other
> > storefiles's
> > > > > bloom filter info to determine
> > > > > whether we can remove it safely or not.
> > > > > I am not familiar with compaction&bloomfilter internal, it's just a
> > raw
> > > > > thought right now:)
> > > > >
> > > > > Regards,
> > > > > Liang
> > > >
> > >
> >
>
+
lars hofhansl 2013-03-14, 18:05