Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> When does HBase set modification timestamp of a HFile?


Copy link to this message
-
Re: When does HBase set modification timestamp of a HFile?
On Sat, Oct 20, 2012 at 1:36 PM, yun peng <[EMAIL PROTECTED]> wrote:
> I am trying to understand how and when hbase set the modification
> timestamp for hfiles. My original intention is to get a timestamp when
> a hfile is generated (when last write to a hfile in compaction).
> StoreFile.getModificationTime() looks a good candidate but after
> initial tests, it has some behaviour that confuses.
>
> My test case is like,
> @hbase shell
> put 'usertable', "key1", 'cf:c1', "v1"
> put 'usertable', "key1", 'cf:c1', "v2"
> get 'usertable', 'key1', {COLUMN => 'cf:c1', VERSIONS => 4}
> flush 'usertable'
> major_compact 'usertable'
>
> for the get operation, it echoes
> COLUMN             CELL
>  cf:c1             timestamp=1350764448150, value=v2
>  cf:c1             timestamp=1350764448114, value=v1
>
> but when I try to get modification timestamp of the hfile generated by
> major_compact, it is 1350764448000, which is smaller/earlier than that
> of the keyvalues (which are actually put first). I have run the same
> test this couple of times, and it is not always: sometime modification
> timestamp is eariler sometimes it's later than keyvalue's.
>
> Anyone knows how HBase set modification timestamp of hfile and that of
> a keyvalue pair? Or generally, how should I get a timestamp indicating
> when the last write to a hfile occurs?
>

This is an interesting project.  Is this an hbase running on a
distributed hdfs?  Is the mod time set when file is opened?  I've not
dug in but should be easy enough figuring when the file gets its mod
time.  If you see variety in how its being set, that would be
particularly interesting.  What versions of hbase+hdfs?

Thanks,
St.Ack