Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> Does hbase.hregion.max.filesize have a limit?


+
Cheng Su 2012-11-01, 05:22
+
lars hofhansl 2012-11-01, 05:35
+
Cheng Su 2012-11-01, 06:45
+
Jeremy Carroll 2012-11-01, 16:39
Copy link to this message
-
Re: Does hbase.hregion.max.filesize have a limit?
Thank you all guys.

I found out that I misunderstood the "size of a region" and "size of a
region server".
I found this property
193-  <property>
194-    <name>hbase.regionserver.regionSplitLimit</name>
195-    <value>2147483647</value>
196-    <description>Limit for the number of regions after which no more region
197:    splitting should take place. This is not a hard limit for the number of
198:    regions but acts as a guideline for the regionserver to stop
splitting after
199:    a certain limit. Default is set to MAX_INT; i.e. do not block splitting.
200-    </description>
201-  </property>

So in practice, a region server can handle enough regions, so I don't
need worry about the store size.

Thank you all again.

On Fri, Nov 2, 2012 at 12:39 AM, Jeremy Carroll <[EMAIL PROTECTED]> wrote:
> To speak to 'if it's possible', yes it is. We have some tables over here at
> Klout during testing where we set the max region size to 100Gb, and
> actually had tables of that size during a MR job that created HFileV2's for
> us to import. So I can say that I have seen 100Gb regions that still work.
>
> As to if this is a good idea, it's probably not. As a capacity planning
> exercise we added additional nodes to the cluster, and split these regions
> down to 10-20Gb in size.
>
> On Wed, Oct 31, 2012 at 11:45 PM, Cheng Su <[EMAIL PROTECTED]> wrote:
>
>> Does that means the max file size of 1 cf is 20G? If I have 3 region
>> servers, then 60G total?
>> I have a very large table, size of one cf (contains only one column)
>> may exceed 60G.
>> Is there any chance to store the data without increase machines?
>>
>> Can multiple region servers runs on one real machine?
>> (I guess not though)
>>
>> On Thu, Nov 1, 2012 at 1:35 PM, lars hofhansl <[EMAIL PROTECTED]> wrote:
>> > The tribal knowledge would say about 20G is the max.
>> > The fellas from Facebook will have a more definite answer.
>> >
>> > -- Lars
>> >
>> >
>> >
>> > ________________________________
>> >  From: Cheng Su <[EMAIL PROTECTED]>
>> > To: [EMAIL PROTECTED]
>> > Sent: Wednesday, October 31, 2012 10:22 PM
>> > Subject: Does hbase.hregion.max.filesize have a limit?
>> >
>> > Hi, all.
>> >
>> > I have a simple question: does hbase.hregion.max.filesize have a limit?
>> > May I specify a very large value to this? like 40G or more? (don't
>> > consider the performance)
>> > I didn't find any description about this from official site or google.
>> >
>> > Thanks.
>> >
>> > --
>> >
>> > Regards,
>> > Cheng Su
>>
>>
>>
>> --
>>
>> Regards,
>> Cheng Su
>>

--

Regards,
Cheng Su
+
ramkrishna vasudevan 2012-11-01, 07:05
+
Cheng Su 2012-11-01, 08:09
+
Doug Meil 2012-11-01, 13:26
+
Kevin Odell 2012-11-01, 13:53