Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Calculating size in HBase.

Copy link to this message
Re: Calculating size in HBase.
If you get KeyValue object by the client, then you have its size for free
(it's the size of its internal byte array) - so why is it time consuming?
On Fri, Jul 5, 2013 at 2:23 AM, Bikash Agrawal <[EMAIL PROTECTED]>wrote:

> Hi ,
> Need to calculate data size in HBase. I can do it by using KV length. But
> it is time consuming with huge data block.
> HFile looks better solution but still I have question with HFile, HFile
> will give size of data block. What if I have limit in scan[ ] object start
> row and end row. And those range can be some where in two different HFile.
> To calculate exact size I need to use loop, which is again inefficient.
> Actually, I am trying to run map reduce using HBase as source. I need to
> know what is data size. I am trying to do some performance evaluation.
> Still I can do some approximation calculation, looking at map-reduce input.
> It is always better to know exact size.
> Do you know how can do calculate data size in HBASE.
> --
> With Best Regards
> Bikash Agrawal
> Web/Software Developer
> Mobile: +47 92502701
> www.bikashagrawal.com.np