Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # dev - Confusion with Actual Number of Regions in HBase


Copy link to this message
-
Re: Confusion with Actual Number of Regions in HBase
Ted Yu 2013-02-21, 00:08
What do you see when you use 'ps aux | grep <pid>' command ?
A sample is shown below:

USER              PID  %CPU %MEM      VSZ    RSS   TT  STAT STARTED
 TIME COMMAND
tyu             16468 232.2  1.4  4974136 240528 s002  R+    4:05PM
2:11.15
/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home/bin/javadoc
-J-Xmx2048m @options @packages

Please give us the VSZ and RSS readings.

Thanks

On Wed, Feb 20, 2013 at 3:56 PM, Joarder KAMAL <[EMAIL PROTECTED]> wrote:

> Sorry may be I didn't clarify the question properly.
>
> Q2. As per the default setting maxHeapMB size is 991, so in saturation
> condition HBase can't use more RAM (even if it is available, for example
> the system memory size is 4GB) than this defined size, is it right?
>
>
> Many thanks again...
>
> On 21 February 2013 10:48, Ted Yu <[EMAIL PROTECTED]> wrote:
>
> > I am not familiar with monitoring tools on EC2.
> >
> > You can use Ganglia to monitor hadoop / HBase.
> >
> > Cheers
> >
> > On Wed, Feb 20, 2013 at 3:42 PM, Joarder KAMAL <[EMAIL PROTECTED]>
> wrote:
> >
> > > Hi Ted,
> > >
> > > Thanks for the reply.
> > >
> > > Q1. Cleared now. Sorry I didn't count the -ROOT- and .META. tables
> > > Q2. Not clear. Could explain a bit more? I am using the default
> setting.
> > >
> > > Thanks again.
> > >
> > >
> > > On 21 February 2013 10:36, Ted Yu <[EMAIL PROTECTED]> wrote:
> > >
> > > > bq. but when I click to view the table details it shows there are 5
> > > > regions.
> > > >
> > > > Did you include -ROOT- and .META. tables above ?
> > > >
> > > > bq. as you can see the maxHeapMB size is 991
> > > >
> > > > Attachment is automatically stripped from your email. FYI
> > > >
> > > > On Wed, Feb 20, 2013 at 3:26 PM, Joarder KAMAL <[EMAIL PROTECTED]>
> > > wrote:
> > > >
> > > > > Dear All,
> > > > >
> > > > > I am currently running a 3 node HBase cluster with Amazon EC2 small
> > > > > instance (1CPU, 1.7GB Mem). After running a simple workload (type
> A)
> > > > using
> > > > > YCSB benchmarking tool, I noticed in the Web UI that the number of
> > > > regions
> > > > > in the 3 region servers are 7 but when I click to view the table
> > > details
> > > > it
> > > > > shows there are 5 regions. I am a bit confuse. Does the Web UI
> > showing
> > > > > wrong results? How can I check region details from the shell/is
> there
> > > any
> > > > > other command line tools for that? I am using Hbase 0.94.3 (as it
> is
> > > the
> > > > > latest version compatible with YCSB at this moment) with
> > Hadoop-1.0.4.
> > > > >
> > > > > Another thing, as you can see the maxHeapMB size is 991 (default
> > > setting)
> > > > > then whenever I am running workloads the server free memory status
> is
> > > > > terrible - almost 0MB for 1M/10M transactions (YCSB/Workload-A). My
> > > > > question is if the maxHeapMB is fixed at 1GB then why I am loosing
> > > extra
> > > > > 700MB memory? How can I monitor cpu/memory/io usage for individual
> > > > cluster
> > > > > components (HBase/Hadoop/MapRed/ZK, etc.)? Is there any tool
> > available
> > > > with
> > > > > HBase?
> > > > >
> > > > > Thanks a lot.
> > > > >
> > > > >
> > > > > [image: Inline images 1]
> > > > >
> > > > >
> > > > > Regards,
> > > > > Joarder Kamal
> > > > >
> > > > > ------------------------------------------
> > > > > PhD Student
> > > > > Gippsland School of Information Technology,
> > > > > Faculty of Information Technology,
> > > > > MONASH University
> > > > > Churchill VIC, Australia. 3842
> > > > >
> > > > >
> > > >
> > >
> >
>