Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> question about Hregion.incrementColumnValue


Copy link to this message
-
RE: question about Hregion.incrementColumnValue
How does doing currentTimeMillis() twice in a row guarantee different timestamps?  And in this case, we're talking about the MemStore vs. HLog not HFile.

There is another section of the code where there is a timestamp+1 to avoid duplicates but this is something else.

> -----Original Message-----
> From: Ryan Rawson [mailto:[EMAIL PROTECTED]]
> Sent: Monday, January 10, 2011 2:27 PM
> To: [EMAIL PROTECTED]
> Subject: Re: question about Hregion.incrementColumnValue
>
> I put more comments on this:
> HBASE-3021
>
> Basically we needed to avoid duplicate timestamp KVs in memstore & hfile,
> elsewise we might end up 'getting' the wrong value and thus messing up the
> count.
>
> With work on ACID by stack we can avoid using that.
>
> -ryan
>
> On Mon, Jan 10, 2011 at 11:23 AM, Stack <[EMAIL PROTECTED]> wrote:
> > Yeah, thats going away unless Ryan comes up w/ a reason for why we
> > should keep it.
> > St.Ack
> >
> > On Mon, Jan 10, 2011 at 12:29 AM, Ryan Rawson <[EMAIL PROTECTED]>
> wrote:
> >> There was a good, but complex reason. Its going away with stacks time
> >> stamp patch. I'll see if I can do a better email tomorrow.
> >> On Jan 9, 2011 11:42 PM, "Dhruba Borthakur" <[EMAIL PROTECTED]>
> wrote:
> >>> I am looking at Hregion.incrementColumnValue(). It has the following
> >>> piece of code
> >>>
> >>> // build the KeyValue now:
> >>> 3266 KeyValue newKv = new KeyValue(row, family,
> >>> 3267 qualifier, EnvironmentEdgeManager.currentTimeMillis(),
> >>> 3268 Bytes.toBytes(result));
> >>> 3269
> >>> 3270 // now log it:
> >>> 3271 if (writeToWAL) {
> >>> 3272 long now = EnvironmentEdgeManager.currentTimeMillis();
> >>> 3273 WALEdit walEdit = new WALEdit();
> >>> 3274 walEdit.add(newKv);
> >>> 3275 this.log.append(regionInfo,
> >>> regionInfo.getTableDesc().getName(),
> >>> 3276 walEdit, now);
> >>> 3277 }
> >>>
> >>> It invokes EnvironmentEdgeManager.currentTimeMillis() twice, once
> >>> for creating the new KV and then another time to add it to the WAL.
> >>> Is this significant or just an oversight? Can we instead invoke it
> >>> once before we create the new key-value and then use it for both code
> paths?
> >>>
> >>> Thanks,
> >>> dhruba
> >>>
> >>> --
> >>> Connect to me at http://www.facebook.com/dhruba
> >>
> >
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB