Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Hmaster is OutOfMemory

Copy link to this message
Re: Hmaster is OutOfMemory
> On May 12, 2011, at 2:24 AM, Gaojinchao <[EMAIL PROTECTED]> wrote:
> I am afraid of some problem:
> 1, One region server has about 12k reigons or more, if up the hregion.max.filesize , parallel scalability will be low and affect scan latency .
> 2, If a region server crashed, It could bring on the other region server crashed?
> Can you give some suggestion about the hbase parameter and my cluster ?

From what everyone I've talked with, a rule of thumb is to have no more than 1000 regions per region server.
You can set this on a per table basis...

If you do the math, a 2GB max file size would mean that each node would then be able to have 2TB of hbase storage within 1000 regions per region server.

I'm testing this... But it takes a while to generate the data for the test...
The goal is to see how 4GB files work and to see the impact on the cluster.
At 4GB max files... 1000 regions would be 4 TB per node ...

Note YMMV and I'm still testing. I'm sure someone with more experience has some better advice...
Sent from a remote device. Please excuse any typos...

Mike Segel