Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - What happened in hlog if data are deleted cuased by ttl?


Copy link to this message
-
RE: What happened in hlog if data are deleted cuased by ttl?
Ramkrishna.S.Vasudevan 2012-08-16, 04:16
Hi

Just to add on,  The HLog is just an edit log.  Any transaction updates(
Puts/Deletes) are just added to HLog.  It is the Scanner that takes care of
the TTL part which is calculated from the TTL configured at the column
family(Store) level.

Regards
Ram

> -----Original Message-----
> From: Harsh J [mailto:[EMAIL PROTECTED]]
> Sent: Tuesday, August 14, 2012 8:51 PM
> To: [EMAIL PROTECTED]
> Subject: Re: What happened in hlog if data are deleted cuased by ttl?
>
> Yes, TTL deletions are done only during compactions. They aren't
> "Deleted" in the sense of what a Delete insert signifies, but are
> rather eliminated in the write process when new
> storefiles are written out - if the value being written to the
> compacted store has already expired.
>
> On Tue, Aug 14, 2012 at 8:40 PM, yonghu <[EMAIL PROTECTED]> wrote:
> > Hi Hars,
> >
> > Thanks for your reply. If I understand you right, it means the ttl
> > deletion will not reflect in log.
> >
> > On Tue, Aug 14, 2012 at 3:24 PM, Harsh J <[EMAIL PROTECTED]> wrote:
> >> Hi Yonghu,
> >>
> >> A timestamp is stored along with each insert. The ttl is maintained
> at
> >> the region-store level. Hence, when the log replays, all entries
> with
> >> expired TTLs are automatically omitted.
> >>
> >> Also, TTL deletions happen during compactions, and hence do not
> >> carry/need Delete events. When scanning a store file, TTL-expired
> >> entries are automatically skipped away.
> >>
> >> On Tue, Aug 14, 2012 at 3:34 PM, yonghu <[EMAIL PROTECTED]>
> wrote:
> >>> My hbase version is 0.92. I tried something as follows:
> >>> 1.Created a table 'test' with 'course' in which ttl=5.
> >>> 2. inserted one row into the table. 5 seconds later, the row was
> deleted.
> >>> Later when I checked the log infor of 'test' table, I only found
> the
> >>> inserted information but not deleted information.
> >>>
> >>> Can anyone tell me which information is written into hlog when data
> is
> >>> deleted by ttl or in this situation, no information is written into
> >>> the hlog. If there is no information of deletion in the log, how
> can
> >>> we guarantee the data recovered by log are correct?
> >>>
> >>> Thanks!
> >>>
> >>> Yong
> >>
> >>
> >>
> >> --
> >> Harsh J
>
>
>
> --
> Harsh J