Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # dev >> Consistency issue when a Put is in the memstore but a more recent Delete is cleaned in a major compaction


+
Jean-Daniel Cryans 2013-03-19, 20:42
+
Ted Yu 2013-03-19, 21:24
+
Ted Yu 2013-03-19, 21:27
+
Enis Söztutar 2013-03-19, 21:32
+
Sergey Shelukhin 2013-03-20, 01:36
Copy link to this message
-
Re: Consistency issue when a Put is in the memstore but a more recent Delete is cleaned in a major compaction
As long as a client application can date a Delete into the future or a Put into the past I do not see how we can eliminate these conditions completely and not at the same time also give up idempotent updates (which is a great property of HBase).
________________________________
 From: Sergey Shelukhin <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]
Sent: Tuesday, March 19, 2013 6:36 PM
Subject: Re: Consistency issue when a Put is in the memstore but a more recent Delete is cleaned in a major compaction
 
Yes, it is the same window of opportunity that was mentioned in some other
JIRAs and threads (see also
HBASE-7902<https://issues.apache.org/jira/browse/HBASE-7902>)
and would have been made wider by dropping deletes during minor compactions
:)
If flush is done before major compaction, the window will just be narrowed,
not eliminated, because you could write data during major compaction...

On Tue, Mar 19, 2013 at 2:32 PM, Enis Söztutar <[EMAIL PROTECTED]> wrote:

> I think this came up in recent discussions about the whether we get get rid
> of delete tombstones on non-major compactions. One of the subtasks for
> stripe compactions deals with this case.
>
>
> On Tue, Mar 19, 2013 at 2:27 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
> > This is also related:
> >
> > HBASE-8086 Major compact should flush memstore firstly
> >
> > On Tue, Mar 19, 2013 at 2:24 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
> >
> > > Here is one related thread (on minor compaction, though) :
> > >
> > > Is it feasible to delete qualified tombstones during minor compaction?
> > >
> > >
> > > On Tue, Mar 19, 2013 at 1:42 PM, Jean-Daniel Cryans <
> [EMAIL PROTECTED]
> > >wrote:
> > >
> > >> Hey guys,
> > >>
> > >> I looked around a bit and couldn't find a jira directly related to
> > >> this. Here's an example of inconsistency in every HBase version
> > >> (although the shell won't let you do it in trunk):
> > >>
> > >> create 't', 'f'
> > >> delete 't', '1', 'f:1', 3
> > >> flush 't'
> > >> put 't', '1', 'f:1', '1', 2
> > >> scan 't'
> > >> major_compact 't'
> > >> scan 't'
> > >>
> > >> The first scan returns nothing, the second one returns the row 1. This
> > >> is the same when the delete is bulk loaded and then major compacted
> > >> (which is a more legitimate use case).
> > >>
> > >> What's the common wisdom here? Does anyone remember if we had this
> > >> discussion in the past?
> > >>
> > >> Thx,
> > >>
> > >> J-D
> > >>
> > >
> > >
> >
>
+
Jean-Daniel Cryans 2013-03-20, 18:01
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB