Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> max HStoreFile size


Copy link to this message
-
Re: max HStoreFile size
Hi Amit,

Yes. You can set split policy per table. Here is relevant part of hbase
book:

http://hbase.apache.org/book/regions.arch.html

The policy can set globally through the HBaseConfiguration used or on a per
table basis:

HTableDescriptor myHtd = ...;
myHtd.setValue(HTableDescriptor.SPLIT_POLICY,
MyCustomSplitPolicy.class.getName());

Cheers

On Thu, Jan 23, 2014 at 9:34 AM, Amit Sela <[EMAIL PROTECTED]> wrote:

> So I think my problem is that from 0.94 the default split policy
> is: IncreasingToUpperBoundRegionSplitPolicy and
> not ConstantSizeRegionSplitPolicy.
>
> Can I set split policy per table ?
>
> Still don't know if hbase.hregion.max.filesize relates to compressed or
> uncompressed ?
>
>
> On Wed, Jan 22, 2014 at 7:25 PM, Amit Sela <[EMAIL PROTECTED]> wrote:
>
> > Hi all, I'm using HBase 0.94.12 and in some tables I'm managing splitting
> > and compactions manually.
> >
> > I was wondering if hbase.hregion.max.filesize relates to compressed or
> > uncompressed file size.
> > If I'm using compression, and the file size < hbase.hregion.max.filesize
> > but uncompressed it's bigger, than when executing major compaction on the
> > region, it splits.
> >
> > Should it be like that ? more important, the recommendation of regions of
> > 1GB is for compressed or uncompressed StoreFile size?
> >
> > Since I'm using bulk load, I get about 3 StoreFiles loaded into each CF
> of
> > every new region, I executed region compaction to unite them as 1 file
> (and
> > then got the unwanted splits) - If I'm never updating this data, do I
> gain
> > something from uniting the files ?
> > Could I manage ~500MB of compressed (GZ - decompresses to about 7.5GB)
> > with 10GB RAM RegionServers ?
> >
> > Thanks,
> >
> > Amit.
> >
> >
> >
> >
> >
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB