Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Does hbase.hregion.max.filesize have a limit?


Copy link to this message
-
Re: Does hbase.hregion.max.filesize have a limit?
Thank you all guys.

I found out that I misunderstood the "size of a region" and "size of a
region server".
I found this property
193-  <property>
194-    <name>hbase.regionserver.regionSplitLimit</name>
195-    <value>2147483647</value>
196-    <description>Limit for the number of regions after which no more region
197:    splitting should take place. This is not a hard limit for the number of
198:    regions but acts as a guideline for the regionserver to stop
splitting after
199:    a certain limit. Default is set to MAX_INT; i.e. do not block splitting.
200-    </description>
201-  </property>

So in practice, a region server can handle enough regions, so I don't
need worry about the store size.

Thank you all again.

On Fri, Nov 2, 2012 at 12:39 AM, Jeremy Carroll <[EMAIL PROTECTED]> wrote:
> To speak to 'if it's possible', yes it is. We have some tables over here at
> Klout during testing where we set the max region size to 100Gb, and
> actually had tables of that size during a MR job that created HFileV2's for
> us to import. So I can say that I have seen 100Gb regions that still work.
>
> As to if this is a good idea, it's probably not. As a capacity planning
> exercise we added additional nodes to the cluster, and split these regions
> down to 10-20Gb in size.
>
> On Wed, Oct 31, 2012 at 11:45 PM, Cheng Su <[EMAIL PROTECTED]> wrote:
>
>> Does that means the max file size of 1 cf is 20G? If I have 3 region
>> servers, then 60G total?
>> I have a very large table, size of one cf (contains only one column)
>> may exceed 60G.
>> Is there any chance to store the data without increase machines?
>>
>> Can multiple region servers runs on one real machine?
>> (I guess not though)
>>
>> On Thu, Nov 1, 2012 at 1:35 PM, lars hofhansl <[EMAIL PROTECTED]> wrote:
>> > The tribal knowledge would say about 20G is the max.
>> > The fellas from Facebook will have a more definite answer.
>> >
>> > -- Lars
>> >
>> >
>> >
>> > ________________________________
>> >  From: Cheng Su <[EMAIL PROTECTED]>
>> > To: [EMAIL PROTECTED]
>> > Sent: Wednesday, October 31, 2012 10:22 PM
>> > Subject: Does hbase.hregion.max.filesize have a limit?
>> >
>> > Hi, all.
>> >
>> > I have a simple question: does hbase.hregion.max.filesize have a limit?
>> > May I specify a very large value to this? like 40G or more? (don't
>> > consider the performance)
>> > I didn't find any description about this from official site or google.
>> >
>> > Thanks.
>> >
>> > --
>> >
>> > Regards,
>> > Cheng Su
>>
>>
>>
>> --
>>
>> Regards,
>> Cheng Su
>>

--

Regards,
Cheng Su
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB