Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> max HStoreFile size


Copy link to this message
-
Re: max HStoreFile size
I set the compaction policy to constant size and still, when compacting
bulk loaded regions, it splits the regions (the region's size is much
smaller than max file size but I do use compression...)
On Jan 23, 2014 12:11 PM, "Samir Ahmic" <[EMAIL PROTECTED]> wrote:

> Hi Amit,
>
> Yes. You can set split policy per table. Here is relevant part of hbase
> book:
>
> http://hbase.apache.org/book/regions.arch.html
>
> The policy can set globally through the HBaseConfiguration used or on a per
> table basis:
>
> HTableDescriptor myHtd = ...;
> myHtd.setValue(HTableDescriptor.SPLIT_POLICY,
> MyCustomSplitPolicy.class.getName());
>
> Cheers
>
>
>
> On Thu, Jan 23, 2014 at 9:34 AM, Amit Sela <[EMAIL PROTECTED]> wrote:
>
> > So I think my problem is that from 0.94 the default split policy
> > is: IncreasingToUpperBoundRegionSplitPolicy and
> > not ConstantSizeRegionSplitPolicy.
> >
> > Can I set split policy per table ?
> >
> > Still don't know if hbase.hregion.max.filesize relates to compressed or
> > uncompressed ?
> >
> >
> > On Wed, Jan 22, 2014 at 7:25 PM, Amit Sela <[EMAIL PROTECTED]> wrote:
> >
> > > Hi all, I'm using HBase 0.94.12 and in some tables I'm managing
> splitting
> > > and compactions manually.
> > >
> > > I was wondering if hbase.hregion.max.filesize relates to compressed or
> > > uncompressed file size.
> > > If I'm using compression, and the file size <
> hbase.hregion.max.filesize
> > > but uncompressed it's bigger, than when executing major compaction on
> the
> > > region, it splits.
> > >
> > > Should it be like that ? more important, the recommendation of regions
> of
> > > 1GB is for compressed or uncompressed StoreFile size?
> > >
> > > Since I'm using bulk load, I get about 3 StoreFiles loaded into each CF
> > of
> > > every new region, I executed region compaction to unite them as 1 file
> > (and
> > > then got the unwanted splits) - If I'm never updating this data, do I
> > gain
> > > something from uniting the files ?
> > > Could I manage ~500MB of compressed (GZ - decompresses to about 7.5GB)
> > > with 10GB RAM RegionServers ?
> > >
> > > Thanks,
> > >
> > > Amit.
> > >
> > >
> > >
> > >
> > >
> >
>