Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> Consistency issue when a Put is in the memstore but a more recent Delete is cleaned in a major compaction


Copy link to this message
-
Re: Consistency issue when a Put is in the memstore but a more recent Delete is cleaned in a major compaction
On Tue, Mar 19, 2013 at 6:50 PM, lars hofhansl <[EMAIL PROTECTED]> wrote:
> As long as a client application can date a Delete into the future or a Put into the past I do not see how we can eliminate these conditions completely and not at the same time also give up idempotent updates (which is a great property of HBase).

The bulk load case doesn't have those properties. The Delete is done
in order, but it gets into an HFile before the Put is flushed.

J-D

>
>
> ________________________________
>  From: Sergey Shelukhin <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]
> Sent: Tuesday, March 19, 2013 6:36 PM
> Subject: Re: Consistency issue when a Put is in the memstore but a more recent Delete is cleaned in a major compaction
>
> Yes, it is the same window of opportunity that was mentioned in some other
> JIRAs and threads (see also
> HBASE-7902<https://issues.apache.org/jira/browse/HBASE-7902>)
> and would have been made wider by dropping deletes during minor compactions
> :)
> If flush is done before major compaction, the window will just be narrowed,
> not eliminated, because you could write data during major compaction...
>
> On Tue, Mar 19, 2013 at 2:32 PM, Enis Söztutar <[EMAIL PROTECTED]> wrote:
>
>> I think this came up in recent discussions about the whether we get get rid
>> of delete tombstones on non-major compactions. One of the subtasks for
>> stripe compactions deals with this case.
>>
>>
>> On Tue, Mar 19, 2013 at 2:27 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>>
>> > This is also related:
>> >
>> > HBASE-8086 Major compact should flush memstore firstly
>> >
>> > On Tue, Mar 19, 2013 at 2:24 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>> >
>> > > Here is one related thread (on minor compaction, though) :
>> > >
>> > > Is it feasible to delete qualified tombstones during minor compaction?
>> > >
>> > >
>> > > On Tue, Mar 19, 2013 at 1:42 PM, Jean-Daniel Cryans <
>> [EMAIL PROTECTED]
>> > >wrote:
>> > >
>> > >> Hey guys,
>> > >>
>> > >> I looked around a bit and couldn't find a jira directly related to
>> > >> this. Here's an example of inconsistency in every HBase version
>> > >> (although the shell won't let you do it in trunk):
>> > >>
>> > >> create 't', 'f'
>> > >> delete 't', '1', 'f:1', 3
>> > >> flush 't'
>> > >> put 't', '1', 'f:1', '1', 2
>> > >> scan 't'
>> > >> major_compact 't'
>> > >> scan 't'
>> > >>
>> > >> The first scan returns nothing, the second one returns the row 1. This
>> > >> is the same when the delete is bulk loaded and then major compacted
>> > >> (which is a more legitimate use case).
>> > >>
>> > >> What's the common wisdom here? Does anyone remember if we had this
>> > >> discussion in the past?
>> > >>
>> > >> Thx,
>> > >>
>> > >> J-D
>> > >>
>> > >
>> > >
>> >
>>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB