Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - Calculating size in HBase.


Copy link to this message
-
Re: Calculating size in HBase.
shashwat shriparv 2013-07-05, 11:44
hadoop dfs -dus /hbase

*Thanks & Regards    *


Shashwat Shriparv

On Fri, Jul 5, 2013 at 4:54 PM, Amit Mor <[EMAIL PROTECTED]> wrote:

> From hadooo shell man
>
> "DU
> Usage: hadoop fs -du URI [URI …]
>
> Displays aggregate length of files contained in the directory or the length
> of a file in case its just a file. Example: hadoop fs -du /user/hadoop/dir1
> /user/hadoop/file1 hdfs://nn.example.com/user/hadoop/dir1 Exit Code:
> Returns 0 on success and -1 on error.
>
> dus Usage: hadoop fs -dus <args>
>
> Displays a summary of file lengths."
>
> You just have to supply your hbase' hdfs path
> On Jul 5, 2013 6:55 AM, "Bikash Agrawal" <[EMAIL PROTECTED]> wrote:
>
> > Hi ,
> >
> > Need to calculate data size in HBase. I can do it by using KV length. But
> > it is time consuming with huge data block.
> > HFile looks better solution but still I have question with HFile, HFile
> > will give size of data block. What if I have limit in scan[ ] object
> start
> > row and end row. And those range can be some where in two different
> HFile.
> > To calculate exact size I need to use loop, which is again inefficient.
> > Actually, I am trying to run map reduce using HBase as source. I need to
> > know what is data size. I am trying to do some performance evaluation.
> > Still I can do some approximation calculation, looking at map-reduce
> input.
> > It is always better to know exact size.
> >
> > Do you know how can do calculate data size in HBASE.
> >
> >
> >
> >
> >
> > --
> > With Best Regards
> > Bikash Agrawal
> > Web/Software Developer
> > Mobile: +47 92502701
> > www.bikashagrawal.com.np
> >
>