Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> Maximum number of tables ?


+
Adrien Mogenet 2012-07-13, 07:04
+
N Keywal 2012-07-13, 08:14
+
Michael Segel 2012-07-13, 15:42
+
Amandeep Khurana 2012-07-13, 15:50
+
Kevin Odell 2012-07-13, 17:36
+
Michael Segel 2012-07-13, 17:40
Copy link to this message
-
Re: Maximum number of tables ?
It is basically unset:

    this.regionSplitLimit = conf.getInt("hbase.regionserver.regionSplitLimit",
        Integer.MAX_VALUE);

(from CompactSplitThread.java).

The number of regions is OK until you dilute the available heap share too much. So you can have >1000 regions (given the block index, file handles etc. keep up) but only a few them can be active most of the time.

Lars

On Jul 13, 2012, at 7:40 PM, Michael Segel wrote:

> I'm going from memory. There was a hardcoded number. I'd have to go back and try to find it.
>
> From a practical standpoint, going over 1000 regions per RS will put you on thin ice.
>
> Too many regions can kill your system.
>
> On Jul 13, 2012, at 12:36 PM, Kevin O'dell wrote:
>
>> Mike,
>>
>> I just saw a system with 2500 Regions per RS(crazy I know, we are fixing
>> that).  I did not think there was a hard coded limit...
>>
>> On Fri, Jul 13, 2012 at 11:50 AM, Amandeep Khurana <[EMAIL PROTECTED]> wrote:
>>
>>> I have come across clusters with 100s of tables but that typically is
>>> due to a sub optimal table design.
>>>
>>> The question here is - why do you need to distribute your data over
>>> lots of tables? What's your access pattern and what kind of data are
>>> you putting in? Or is this just a theoretical question?
>>>
>>> On Jul 13, 2012, at 12:05 AM, Adrien Mogenet <[EMAIL PROTECTED]>
>>> wrote:
>>>
>>>> Hi there,
>>>>
>>>> I read some good practices about number of columns / column families, but
>>>> nothing about the number of tables.
>>>> What if I need to spread my data among hundred or thousand (big) tables ?
>>>> What should I care about ? I guess I should keep a tight number of
>>>> storeFiles per RegionServer ?
>>>>
>>>> --
>>>> Adrien Mogenet
>>>> http://www.mogenet.me
>>>
>>
>>
>>
>> --
>> Kevin O'Dell
>> Customer Operations Engineer, Cloudera
>
+
Adrien Mogenet 2012-07-14, 06:27
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB