Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # dev - memstore timestamp and visible timestamp


Copy link to this message
-
Re: memstore timestamp and visible timestamp
Wei Tan 2012-08-03, 14:21
Hi Lars,

   Appreciate your reply. Actually I read your blog posting and then had
that question. I am very interested in how you guarantee this:

   Also note that if you use the Region Server assigned TSs then mTS1<mTS2
implies TS1<=TS2 (the update might happen with the same ms).

  In case you have a pointer explaining this, I would like to read.
Otherwise I will dig into the code later today. I remember reading 0.92.0
code and do not find much clue. But I will try again.

 

Best Regards,
Wei

Wei Tan
Research Staff Member
IBM T. J. Watson Research Center
19 Skyline Dr, Hawthorne, NY  10532
[EMAIL PROTECTED]; 914-784-6752

From:   lars hofhansl <[EMAIL PROTECTED]>
To:     "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>,
"[EMAIL PROTECTED]" <[EMAIL PROTECTED]>,
Date:   08/02/2012 07:35 PM
Subject:        Re: memstore timestamp and visible timestamp

Hi Wei,

you have to distinguish between "visible to other concurrent scanners" and
"visible to a client".
What's visible to a client is determined by what the a client wants to see
based on the application visible timestamp (TS).

The visibility to concurrent scanners is controlled by the memstoreTS
(mTS) to avoid "strange" states sue to parallel updates.
HBase here guards against partially visible "transactions" (i.e. a Put of
many columns that fails after it applied the changes to some of the
columns).

The scenario you describe below is indeed desired. Note that a client can
request seeing the older versions too so the older edit (in terms of TS is
not lost).
Also note that if you use the Region Server assigned TSs then mTS1<mTS2
implies TS1<=TS2 (the update might happen with the same ms).

If you do not mind a longer read, I have written about this here:
http://hadoop-hbase.blogspot.com/2012/03/acid-in-hbase.html

Let me know if that makes any sense.

-- Lars
----- Original Message -----
From: Wei Tan <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]
Cc:
Sent: Thursday, August 2, 2012 3:35 PM
Subject: memstore timestamp and visible timestamp

Hi,

  I have a question regarding the correlation between the visible
timestamp of a KV (denoted as ts) and its memstore timestamp (aka, the
write number, denoted as memts). Reading the HRegion.java code it seems
that these two are independently assigned. Let's assume two concurrent
put: (k, v1) and (k, v2)
  Suppose somehow memts(k,v1) < memts(k, v2) then (k,v1) will be committed

and visible before (k,v2).
If ts(k,v1) < ts(k, v2), then after both KVs commits, (k,v2) becomes the
latest version.
else, if ts(k,v1) > ts(k, v2), then after a "later"(w.r.t. MVCC) KV
commits, it immediately become stale and still not visible. --- Is it a
desirable feature?
  Am I understanding it correctly, that memts(k,v1) < memts(k, v2) does
not indicate that ts(k,v1) < ts(k, v2), and vice versa?
PS: let's talk about the hbase region server assigned, not user assigned,
visible timestamp.

  Thanks,

Wei

Best Regards,
Wei

Wei Tan
Research Staff Member
IBM T. J. Watson Research Center
19 Skyline Dr, Hawthorne, NY  10532
[EMAIL PROTECTED]; 914-784-6752