Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - max HStoreFile size


Copy link to this message
-
Re: max HStoreFile size
Samir Ahmic 2014-01-23, 10:11
Hi Amit,

Yes. You can set split policy per table. Here is relevant part of hbase
book:

http://hbase.apache.org/book/regions.arch.html

The policy can set globally through the HBaseConfiguration used or on a per
table basis:

HTableDescriptor myHtd = ...;
myHtd.setValue(HTableDescriptor.SPLIT_POLICY,
MyCustomSplitPolicy.class.getName());

Cheers

On Thu, Jan 23, 2014 at 9:34 AM, Amit Sela <[EMAIL PROTECTED]> wrote:

> So I think my problem is that from 0.94 the default split policy
> is: IncreasingToUpperBoundRegionSplitPolicy and
> not ConstantSizeRegionSplitPolicy.
>
> Can I set split policy per table ?
>
> Still don't know if hbase.hregion.max.filesize relates to compressed or
> uncompressed ?
>
>
> On Wed, Jan 22, 2014 at 7:25 PM, Amit Sela <[EMAIL PROTECTED]> wrote:
>
> > Hi all, I'm using HBase 0.94.12 and in some tables I'm managing splitting
> > and compactions manually.
> >
> > I was wondering if hbase.hregion.max.filesize relates to compressed or
> > uncompressed file size.
> > If I'm using compression, and the file size < hbase.hregion.max.filesize
> > but uncompressed it's bigger, than when executing major compaction on the
> > region, it splits.
> >
> > Should it be like that ? more important, the recommendation of regions of
> > 1GB is for compressed or uncompressed StoreFile size?
> >
> > Since I'm using bulk load, I get about 3 StoreFiles loaded into each CF
> of
> > every new region, I executed region compaction to unite them as 1 file
> (and
> > then got the unwanted splits) - If I'm never updating this data, do I
> gain
> > something from uniting the files ?
> > Could I manage ~500MB of compressed (GZ - decompresses to about 7.5GB)
> > with 10GB RAM RegionServers ?
> >
> > Thanks,
> >
> > Amit.
> >
> >
> >
> >
> >
>