Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # user - Does HBase RegionServer benefit from OS Page Cache


+
Pankaj Gupta 2013-03-20, 07:34
+
Jean-Daniel Cryans 2013-03-20, 17:30
+
Enis Söztutar 2013-03-21, 20:28
+
Pankaj Gupta 2013-03-24, 00:31
+
Ted Yu 2013-03-24, 01:13
+
Liyin Tang 2013-03-24, 04:44
+
lars hofhansl 2013-03-24, 05:20
+
Liyin Tang 2013-03-25, 05:15
+
Enis Söztutar 2013-03-25, 18:24
+
Liyin Tang 2013-03-25, 19:18
+
Enis Söztutar 2013-03-25, 20:26
+
谢良 2013-03-26, 03:01
Copy link to this message
-
Re: Does HBase RegionServer benefit from OS Page Cache
Andrew Purtell 2013-03-25, 18:42
> With very large heaps, maybe keeping around the compressed blocks in a
secondary cache makes sense?

That's an interesting idea.

> A compaction will trash the cache. But maybe we can track keyvalues (inside
cached blocks are cached) for the files in the compaction, and mark the
blocks of the resulting compacted file which contain previously cached
keyvalues
to be cached after the compaction.

With very large heaps and a GC that can handle them (perhaps the G1 GC),
another option which might be worth experimenting with is a value (KV)
cache independent of the block cache which could be enabled on a per-table
basis. This would not be trashed by compaction, though we'd need to do some
additional housekeeping to evict deleted cells from the value cache, and
could be useful if collectively RAM on the cluster is sufficient to hold
the whole working set in memory (for the selected tables).
On Mon, Mar 25, 2013 at 7:24 PM, Enis Söztutar <[EMAIL PROTECTED]> wrote:

> Thanks Liyin for sharing your use cases.
>
> Related to those, I was thinking of two improvements:
>  - AFAIK, MySQL keeps the compressed and uncompressed versions of the blocs
> in its block cache, failing over the compressed one if decompressed one
> gets evicted. With very large heaps, maybe keeping around the compressed
> blocks in a secondary cache makes sense?
>  - A compaction will trash the cache. But maybe we can track keyvalues
> (inside cached blocks are cached) for the files in the compaction, and mark
> the blocks of the resulting compacted file which contain previously cached
> keyvalues to be cached after the compaction. I have to research the
> feasibility of this approach.
>
> Enis
>
>
> On Sun, Mar 24, 2013 at 10:15 PM, Liyin Tang <[EMAIL PROTECTED]> wrote:
>
> > Block cache is for uncompressed data while OS page contains the
> compressed
> > data. Unless the request pattern is full-table sequential scan, the block
> > cache is still quite useful. I think the size of the block cache should
> be
> > the amont of hot data we want to retain within a compaction cycle, which
> is
> > quite hard to estimate in some use cases.
> >
> >
> > Thanks a lot
> > Liyin
> > ________________________________________
> > From: lars hofhansl [[EMAIL PROTECTED]]
> > Sent: Saturday, March 23, 2013 10:20 PM
> > To: [EMAIL PROTECTED]
> > Subject: Re: Does HBase RegionServer benefit from OS Page Cache
> >
> > Interesting.
> >
> > > 2) The blocks in the block cache will be naturally invalid quickly
> after
> > the compactions.
> >
> > Should one keep the block cache small in order to increase the OS page
> > cache?
> >
> > Does you data suggest we should not use the block cache at all?
> >
> >
> > Thanks.
> >
> > -- Lars
> >
> >
> >
> > ________________________________
> >  From: Liyin Tang <[EMAIL PROTECTED]>
> > To: [EMAIL PROTECTED]
> > Sent: Saturday, March 23, 2013 9:44 PM
> > Subject: Re: Does HBase RegionServer benefit from OS Page Cache
> >
> > We (Facebook) are closely monitoring the OS page cache hit ratio in the
> > production environments. My experience is if your data access pattern is
> > very random, then the OS page cache won't help you so much even though
> the
> > data locality is very high. On the other hand, if the requests are always
> > against the recent data points, then the page cache hit ratio could be
> much
> > higher.
> >
> > Actually, there are lots of optimizations could be done in HDFS. For
> > example, we are working on fadvice away the 2nd/3rd replicated data from
> OS
> > page cache so that it potentially could improve your OS page cache by 3X.
> > Also, by taking advantage of the tiered-based compaction+fadvice in HDFS,
> > the region server could keep more hot data in OS page cache based on the
> > read access pattern.
> >
> > Another separate point is that we probably should NOT reply on the
> > memstore/block cache to keep hot data. 1) The more data in the memstore,
> > the more data the region server need to recovery from the server

Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)