Cheng Su 2012-11-01, 05:22
lars hofhansl 2012-11-01, 05:35
Cheng Su 2012-11-01, 06:45
Jeremy Carroll 2012-11-01, 16:39
-Re: Does hbase.hregion.max.filesize have a limit?
Cheng Su 2012-11-02, 01:20
Thank you all guys.
I found out that I misunderstood the "size of a region" and "size of a
I found this property
196- <description>Limit for the number of regions after which no more region
197: splitting should take place. This is not a hard limit for the number of
198: regions but acts as a guideline for the regionserver to stop
199: a certain limit. Default is set to MAX_INT; i.e. do not block splitting.
So in practice, a region server can handle enough regions, so I don't
need worry about the store size.
Thank you all again.
On Fri, Nov 2, 2012 at 12:39 AM, Jeremy Carroll <[EMAIL PROTECTED]> wrote:
> To speak to 'if it's possible', yes it is. We have some tables over here at
> Klout during testing where we set the max region size to 100Gb, and
> actually had tables of that size during a MR job that created HFileV2's for
> us to import. So I can say that I have seen 100Gb regions that still work.
> As to if this is a good idea, it's probably not. As a capacity planning
> exercise we added additional nodes to the cluster, and split these regions
> down to 10-20Gb in size.
> On Wed, Oct 31, 2012 at 11:45 PM, Cheng Su <[EMAIL PROTECTED]> wrote:
>> Does that means the max file size of 1 cf is 20G? If I have 3 region
>> servers, then 60G total?
>> I have a very large table, size of one cf (contains only one column)
>> may exceed 60G.
>> Is there any chance to store the data without increase machines?
>> Can multiple region servers runs on one real machine?
>> (I guess not though)
>> On Thu, Nov 1, 2012 at 1:35 PM, lars hofhansl <[EMAIL PROTECTED]> wrote:
>> > The tribal knowledge would say about 20G is the max.
>> > The fellas from Facebook will have a more definite answer.
>> > -- Lars
>> > ________________________________
>> > From: Cheng Su <[EMAIL PROTECTED]>
>> > To: [EMAIL PROTECTED]
>> > Sent: Wednesday, October 31, 2012 10:22 PM
>> > Subject: Does hbase.hregion.max.filesize have a limit?
>> > Hi, all.
>> > I have a simple question: does hbase.hregion.max.filesize have a limit?
>> > May I specify a very large value to this? like 40G or more? (don't
>> > consider the performance)
>> > I didn't find any description about this from official site or google.
>> > Thanks.
>> > --
>> > Regards,
>> > Cheng Su
>> Cheng Su
ramkrishna vasudevan 2012-11-01, 07:05
Cheng Su 2012-11-01, 08:09
Doug Meil 2012-11-01, 13:26
Kevin O'dell 2012-11-01, 13:53