Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> Consistency issue when a Put is in the memstore but a more recent Delete is cleaned in a major compaction


Copy link to this message
-
Re: Consistency issue when a Put is in the memstore but a more recent Delete is cleaned in a major compaction
On Tue, Mar 19, 2013 at 6:50 PM, lars hofhansl <[EMAIL PROTECTED]> wrote:
> As long as a client application can date a Delete into the future or a Put into the past I do not see how we can eliminate these conditions completely and not at the same time also give up idempotent updates (which is a great property of HBase).

The bulk load case doesn't have those properties. The Delete is done
in order, but it gets into an HFile before the Put is flushed.

J-D

>
>
> ________________________________
>  From: Sergey Shelukhin <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]
> Sent: Tuesday, March 19, 2013 6:36 PM
> Subject: Re: Consistency issue when a Put is in the memstore but a more recent Delete is cleaned in a major compaction
>
> Yes, it is the same window of opportunity that was mentioned in some other
> JIRAs and threads (see also
> HBASE-7902<https://issues.apache.org/jira/browse/HBASE-7902>)
> and would have been made wider by dropping deletes during minor compactions
> :)
> If flush is done before major compaction, the window will just be narrowed,
> not eliminated, because you could write data during major compaction...
>
> On Tue, Mar 19, 2013 at 2:32 PM, Enis Söztutar <[EMAIL PROTECTED]> wrote:
>
>> I think this came up in recent discussions about the whether we get get rid
>> of delete tombstones on non-major compactions. One of the subtasks for
>> stripe compactions deals with this case.
>>
>>
>> On Tue, Mar 19, 2013 at 2:27 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>>
>> > This is also related:
>> >
>> > HBASE-8086 Major compact should flush memstore firstly
>> >
>> > On Tue, Mar 19, 2013 at 2:24 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>> >
>> > > Here is one related thread (on minor compaction, though) :
>> > >
>> > > Is it feasible to delete qualified tombstones during minor compaction?
>> > >
>> > >
>> > > On Tue, Mar 19, 2013 at 1:42 PM, Jean-Daniel Cryans <
>> [EMAIL PROTECTED]
>> > >wrote:
>> > >
>> > >> Hey guys,
>> > >>
>> > >> I looked around a bit and couldn't find a jira directly related to
>> > >> this. Here's an example of inconsistency in every HBase version
>> > >> (although the shell won't let you do it in trunk):
>> > >>
>> > >> create 't', 'f'
>> > >> delete 't', '1', 'f:1', 3
>> > >> flush 't'
>> > >> put 't', '1', 'f:1', '1', 2
>> > >> scan 't'
>> > >> major_compact 't'
>> > >> scan 't'
>> > >>
>> > >> The first scan returns nothing, the second one returns the row 1. This
>> > >> is the same when the delete is bulk loaded and then major compacted
>> > >> (which is a more legitimate use case).
>> > >>
>> > >> What's the common wisdom here? Does anyone remember if we had this
>> > >> discussion in the past?
>> > >>
>> > >> Thx,
>> > >>
>> > >> J-D
>> > >>
>> > >
>> > >
>> >
>>