Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> Regarding memstoreTS in bulkloaded HFiles


Copy link to this message
-
Regarding memstoreTS in bulkloaded HFiles
Hi Devs

            In HFile V2 we have introduced the memstore TS to be getting written to the HFiles. In case of bulk load also, now we are writing a long value as part of every KV. I think in case of the bulk loading there is no meaning for the memstore TS. Can we avoid this?

As of now we are not able to set any Block encoder algo as part of bulk loading. But I have created HBASE-6040 which solves this. I have checked the current available encoder algos but none of them handles the memstoreTS as such. There is a new type of trie encoder issue open. In this it seems it will handle this kind of scenario. Only one long value will get stored as memstoreTS for one block.    Still thes all makes it mandatory that some block encoder scheme to be used.

Do we need to think making the memstoreTS write into the HFile (in version 2) as some way configurable? In case of bulk loading we can turn it OFF. Pls correct me if my understanding is wrong

-Anoop-
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB