Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Accumulo >> mail # user >> Insert and delete row


+
Roshan Punnoose 2013-01-17, 13:59
+
Billie Rinaldi 2013-01-17, 14:40
+
Roshan Punnoose 2013-01-17, 15:50
Copy link to this message
-
Re: Insert and delete row
On Thu, Jan 17, 2013 at 7:50 AM, Roshan Punnoose <[EMAIL PROTECTED]> wrote:

> I have noticed that some user iterators perform checks for if the key
> isDeleted before performing any work. Does that mean that iterators may see
> deleted keys? (I have been writing my iterators assuming that deleted keys
> may make it to my iterators)
>

I believe the iterator javadocs are accurate: "Note that in minor
compaction scope and in non-full major compaction scopes the iterator may
see deletion entries. These entries should be preserved by all iterators
except ones that are strictly scan-time iterators that will never be
configured for the minc or majc scopes. Deletion entries are only removed
during full major compactions."

So, you won't get deletion entries at scan time or during a full major
compaction (when they will be removed).  You will see them for minor
compactions and non-full major compactions, and your iterator needs to
preserve them.  During those scopes you don't have access to all the data
and there could be entries in other files that should be deleted.

Billie

> Roshan Punnoose
> [EMAIL PROTECTED]
>
>
>
> On Jan 17, 2013, at 9:40 AM, Billie Rinaldi <[EMAIL PROTECTED]> wrote:
>
> On Thu, Jan 17, 2013 at 5:59 AM, Roshan Punnoose <[EMAIL PROTECTED]>wrote:
>
>> We had noticed some interesting behavior in our Accumulo instance. When
>> we delete a row and add that exact same row (with the same timestamp) after
>> the delete, the row still does not exist. It makes sense because the delete
>> really just adds a delete row that tells Accumulo to ignore any row that is
>> equal to it. Is there a way to bypass this? Where I could set up my own
>> iterator to decide if a row should be deleted or not? (not sure how I would
>> decide yet, but just seeing if it is possible to bypass the delete logic as
>> is)
>>
>
> If you do a full compaction before re-inserting the entry, the deletion
> entry will be removed.  I believe compacting a range is also sufficient
> (available in newer versions of Accumulo).
>
> Billie
>
>
>
>>
>> I tried to set up an Iterator in priority=1 to see if I could see a
>> delete row, but it did not seem to make it to my iterator.
>>
>> Any ideas?
>>
>> Roshan Punnoose
>> [EMAIL PROTECTED]
>>
>>
>>
>>
>
>
+
Keith Turner 2013-01-17, 14:38
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB