Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> When does HBase set modification timestamp of a HFile?


Copy link to this message
-
When does HBase set modification timestamp of a HFile?
Hi, All,
I am trying to understand how and when hbase set the modification
timestamp for hfiles. My original intention is to get a timestamp when
a hfile is generated (when last write to a hfile in compaction).
StoreFile.getModificationTime() looks a good candidate but after
initial tests, it has some behaviour that confuses.

My test case is like,
@hbase shell
put 'usertable', "key1", 'cf:c1', "v1"
put 'usertable', "key1", 'cf:c1', "v2"
get 'usertable', 'key1', {COLUMN => 'cf:c1', VERSIONS => 4}
flush 'usertable'
major_compact 'usertable'

for the get operation, it echoes
COLUMN             CELL
 cf:c1             timestamp=1350764448150, value=v2
 cf:c1             timestamp=1350764448114, value=v1

but when I try to get modification timestamp of the hfile generated by
major_compact, it is 1350764448000, which is smaller/earlier than that
of the keyvalues (which are actually put first). I have run the same
test this couple of times, and it is not always: sometime modification
timestamp is eariler sometimes it's later than keyvalue's.

Anyone knows how HBase set modification timestamp of hfile and that of
a keyvalue pair? Or generally, how should I get a timestamp indicating
when the last write to a hfile occurs?

regards,
Yun
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB