Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> What happened in hlog if data are deleted cuased by ttl?


Copy link to this message
-
RE: What happened in hlog if data are deleted cuased by ttl?
Hi

Just to add on,  The HLog is just an edit log.  Any transaction updates(
Puts/Deletes) are just added to HLog.  It is the Scanner that takes care of
the TTL part which is calculated from the TTL configured at the column
family(Store) level.

Regards
Ram

> -----Original Message-----
> From: Harsh J [mailto:[EMAIL PROTECTED]]
> Sent: Tuesday, August 14, 2012 8:51 PM
> To: [EMAIL PROTECTED]
> Subject: Re: What happened in hlog if data are deleted cuased by ttl?
>
> Yes, TTL deletions are done only during compactions. They aren't
> "Deleted" in the sense of what a Delete insert signifies, but are
> rather eliminated in the write process when new
> storefiles are written out - if the value being written to the
> compacted store has already expired.
>
> On Tue, Aug 14, 2012 at 8:40 PM, yonghu <[EMAIL PROTECTED]> wrote:
> > Hi Hars,
> >
> > Thanks for your reply. If I understand you right, it means the ttl
> > deletion will not reflect in log.
> >
> > On Tue, Aug 14, 2012 at 3:24 PM, Harsh J <[EMAIL PROTECTED]> wrote:
> >> Hi Yonghu,
> >>
> >> A timestamp is stored along with each insert. The ttl is maintained
> at
> >> the region-store level. Hence, when the log replays, all entries
> with
> >> expired TTLs are automatically omitted.
> >>
> >> Also, TTL deletions happen during compactions, and hence do not
> >> carry/need Delete events. When scanning a store file, TTL-expired
> >> entries are automatically skipped away.
> >>
> >> On Tue, Aug 14, 2012 at 3:34 PM, yonghu <[EMAIL PROTECTED]>
> wrote:
> >>> My hbase version is 0.92. I tried something as follows:
> >>> 1.Created a table 'test' with 'course' in which ttl=5.
> >>> 2. inserted one row into the table. 5 seconds later, the row was
> deleted.
> >>> Later when I checked the log infor of 'test' table, I only found
> the
> >>> inserted information but not deleted information.
> >>>
> >>> Can anyone tell me which information is written into hlog when data
> is
> >>> deleted by ttl or in this situation, no information is written into
> >>> the hlog. If there is no information of deletion in the log, how
> can
> >>> we guarantee the data recovered by log are correct?
> >>>
> >>> Thanks!
> >>>
> >>> Yong
> >>
> >>
> >>
> >> --
> >> Harsh J
>
>
>
> --
> Harsh J
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB