Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Where is code in hbase that physically delete a record?


Copy link to this message
-
RE: Where is code in hbase that physically delete a record?
Hi Yun
Logically deleting a KeyValue data in hbase is performed
> by
> marking tombmarker (by Delete() per records) or setting TTL/max_version
> (per Store). After these actions, however, the physical data are still
> there, somewhere in the system. Physically deleting a record in hbase
> is
> realised by *a scanner to discard a keyvalue data record* during the
> major_compact.
Yes correct.  As you understood correctly the major_compact will try to
avoid the deleted records when the kvs are copied
Into a new file by major compaction.  

In 0.94.2 there are some new hooks added like preCompactScannerOpen where
you can write your own scanner implementation.
But this will help you to write custom logic of which KVs to avoid during
compaction.  For eg, say you don't want any KV where any specific col c1 'a'.
Then you can write your scanner and pass it thro preCompactScanneOpen.

But suppose the system itself is trying to avoid the kvs that got deleted
then currently there is no hook provided in CP to get those values
specifically.

Hope this helps.

Regards
Ram

> -----Original Message-----
> From: yun peng [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, October 17, 2012 5:54 PM
> To: [EMAIL PROTECTED]
> Subject: Where is code in hbase that physically delete a record?
>
> Hi, All,
> I want to find internal code in hbase where physical deleting a record
> occurs.
>
> -some of my understanding.
> Correct me if I am wrong. (It is largely based on my experience and
> even
> speculation.) Logically deleting a KeyValue data in hbase is performed
> by
> marking tombmarker (by Delete() per records) or setting TTL/max_version
> (per Store). After these actions, however, the physical data are still
> there, somewhere in the system. Physically deleting a record in hbase
> is
> realised by *a scanner to discard a keyvalue data record* during the
> major_compact.
>
> -what I need
> I want to extend hbase to associate some actions with physically
> deleting a
> record. Does hbase provide such hook (or coprocessor API) to inject
> code
> for each KV record that is skipped by hbase storescanner in
> major_compact.
> If not, anyone knows where should I look into in hbase (-0.94.2) for
> such
> code modification?
>
> Thanks.
> Yun