Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> Region server block cache and memstore size

Ivan Tretyakov 2013-11-28, 09:46
Copy link to this message
Re: Region server block cache and memstore size
So you use Bulk load with HFileOpFormat for writing data?  Then you can
reduce the hbase.regionserver.global.memstore.upperLimit and
hbase.regionserver.global.memstore.lowerLimit  and give more heap % for the
block cache.  Not getting why u try to reduce that also.

On Thu, Nov 28, 2013 at 3:16 PM, Ivan Tretyakov <[EMAIL PROTECTED]
> wrote:

> Hi!
> We are using HBase 0.92.1-cdh4.1.1. To import data the only way we use is
> bulk load. And our common access pattern is sequential scans of different
> parts of the tables.
> Since that we are considering to disable block cache by setting
> hbase.block.cache.size to zero.
> But We've found following in HBase book (
> http://hbase.apache.org/book/important_configurations.html):
> "Do not turn off block cache (You'd do it by setting hbase.block.cache.size
> to zero). Currently we do not do well if you do this because the
> regionserver will spend all its time loading hfile indices over and over
> again. If your working set it such that block cache does you no good, at
> least size the block cache such that hfile indices will stay up in the
> cache (you can get a rough idea on the size you need by surveying
> regionserver UIs; you'll see index block size accounted near the top of the
> webpage)."
> Another thing we consider to reduce is memstore size by tuning following
> options hbase.regionserver.global.memstore.upperLimit and
> hbase.regionserver.global.memstore.lowerLimit.
> So, my questions are:
> Does it make sense to touch these options in our case?
> Is this memory reserved or other processes inside regionserver can use it?
> Thanks in advance!
> --
> Best Regards
> Ivan Tretyakov
> Deployment Engineer
> Grid Dynamics
> +7 812 640 38 76<https://mail.google.com/mail/u/0/html/compose/static_files/blank_quirks.html#>
> Skype: ivan.v.tretyakov
> www.griddynamics.com
Kevin Odell 2013-11-28, 13:50