Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Region server block cache and memstore size


Copy link to this message
-
Re: Region server block cache and memstore size
So you use Bulk load with HFileOpFormat for writing data?  Then you can
reduce the hbase.regionserver.global.memstore.upperLimit and
hbase.regionserver.global.memstore.lowerLimit  and give more heap % for the
block cache.  Not getting why u try to reduce that also.

-Anoop-
On Thu, Nov 28, 2013 at 3:16 PM, Ivan Tretyakov <[EMAIL PROTECTED]
> wrote:

> Hi!
>
> We are using HBase 0.92.1-cdh4.1.1. To import data the only way we use is
> bulk load. And our common access pattern is sequential scans of different
> parts of the tables.
>
> Since that we are considering to disable block cache by setting
> hbase.block.cache.size to zero.
> But We've found following in HBase book (
> http://hbase.apache.org/book/important_configurations.html):
>
> "Do not turn off block cache (You'd do it by setting hbase.block.cache.size
> to zero). Currently we do not do well if you do this because the
> regionserver will spend all its time loading hfile indices over and over
> again. If your working set it such that block cache does you no good, at
> least size the block cache such that hfile indices will stay up in the
> cache (you can get a rough idea on the size you need by surveying
> regionserver UIs; you'll see index block size accounted near the top of the
> webpage)."
>
> Another thing we consider to reduce is memstore size by tuning following
> options hbase.regionserver.global.memstore.upperLimit and
> hbase.regionserver.global.memstore.lowerLimit.
>
> So, my questions are:
>
> Does it make sense to touch these options in our case?
> Is this memory reserved or other processes inside regionserver can use it?
>
> Thanks in advance!
>
> --
> Best Regards
> Ivan Tretyakov
>
> Deployment Engineer
> Grid Dynamics
> +7 812 640 38 76<https://mail.google.com/mail/u/0/html/compose/static_files/blank_quirks.html#>
> Skype: ivan.v.tretyakov
> www.griddynamics.com
> [EMAIL PROTECTED]
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB