-Re: Region server block cache and memstore size
Anoop John 2013-11-28, 09:50
So you use Bulk load with HFileOpFormat for writing data? Then you can
reduce the hbase.regionserver.global.memstore.upperLimit and
hbase.regionserver.global.memstore.lowerLimit and give more heap % for the
block cache. Not getting why u try to reduce that also.
On Thu, Nov 28, 2013 at 3:16 PM, Ivan Tretyakov <[EMAIL PROTECTED]
> We are using HBase 0.92.1-cdh4.1.1. To import data the only way we use is
> bulk load. And our common access pattern is sequential scans of different
> parts of the tables.
> Since that we are considering to disable block cache by setting
> hbase.block.cache.size to zero.
> But We've found following in HBase book (
> "Do not turn off block cache (You'd do it by setting hbase.block.cache.size
> to zero). Currently we do not do well if you do this because the
> regionserver will spend all its time loading hfile indices over and over
> again. If your working set it such that block cache does you no good, at
> least size the block cache such that hfile indices will stay up in the
> cache (you can get a rough idea on the size you need by surveying
> regionserver UIs; you'll see index block size accounted near the top of the
> Another thing we consider to reduce is memstore size by tuning following
> options hbase.regionserver.global.memstore.upperLimit and
> So, my questions are:
> Does it make sense to touch these options in our case?
> Is this memory reserved or other processes inside regionserver can use it?
> Thanks in advance!
> Best Regards
> Ivan Tretyakov
> Deployment Engineer
> Grid Dynamics
> +7 812 640 38 76<https://mail.google.com/mail/u/0/html/compose/static_files/blank_quirks.html#>
> Skype: ivan.v.tretyakov
> [EMAIL PROTECTED]