Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # dev - Consistency issue when a Put is in the memstore but a more recent Delete is cleaned in a major compaction


+
Jean-Daniel Cryans 2013-03-19, 20:42
+
Ted Yu 2013-03-19, 21:24
+
Ted Yu 2013-03-19, 21:27
+
Enis Söztutar 2013-03-19, 21:32
+
Sergey Shelukhin 2013-03-20, 01:36
Copy link to this message
-
Re: Consistency issue when a Put is in the memstore but a more recent Delete is cleaned in a major compaction
lars hofhansl 2013-03-20, 01:50
As long as a client application can date a Delete into the future or a Put into the past I do not see how we can eliminate these conditions completely and not at the same time also give up idempotent updates (which is a great property of HBase).
________________________________
 From: Sergey Shelukhin <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]
Sent: Tuesday, March 19, 2013 6:36 PM
Subject: Re: Consistency issue when a Put is in the memstore but a more recent Delete is cleaned in a major compaction
 
Yes, it is the same window of opportunity that was mentioned in some other
JIRAs and threads (see also
HBASE-7902<https://issues.apache.org/jira/browse/HBASE-7902>)
and would have been made wider by dropping deletes during minor compactions
:)
If flush is done before major compaction, the window will just be narrowed,
not eliminated, because you could write data during major compaction...

On Tue, Mar 19, 2013 at 2:32 PM, Enis Söztutar <[EMAIL PROTECTED]> wrote:

> I think this came up in recent discussions about the whether we get get rid
> of delete tombstones on non-major compactions. One of the subtasks for
> stripe compactions deals with this case.
>
>
> On Tue, Mar 19, 2013 at 2:27 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
> > This is also related:
> >
> > HBASE-8086 Major compact should flush memstore firstly
> >
> > On Tue, Mar 19, 2013 at 2:24 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
> >
> > > Here is one related thread (on minor compaction, though) :
> > >
> > > Is it feasible to delete qualified tombstones during minor compaction?
> > >
> > >
> > > On Tue, Mar 19, 2013 at 1:42 PM, Jean-Daniel Cryans <
> [EMAIL PROTECTED]
> > >wrote:
> > >
> > >> Hey guys,
> > >>
> > >> I looked around a bit and couldn't find a jira directly related to
> > >> this. Here's an example of inconsistency in every HBase version
> > >> (although the shell won't let you do it in trunk):
> > >>
> > >> create 't', 'f'
> > >> delete 't', '1', 'f:1', 3
> > >> flush 't'
> > >> put 't', '1', 'f:1', '1', 2
> > >> scan 't'
> > >> major_compact 't'
> > >> scan 't'
> > >>
> > >> The first scan returns nothing, the second one returns the row 1. This
> > >> is the same when the delete is bulk loaded and then major compacted
> > >> (which is a more legitimate use case).
> > >>
> > >> What's the common wisdom here? Does anyone remember if we had this
> > >> discussion in the past?
> > >>
> > >> Thx,
> > >>
> > >> J-D
> > >>
> > >
> > >
> >
>
+
Jean-Daniel Cryans 2013-03-20, 18:01