Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - client cache for all region server information?


Copy link to this message
-
Re: client cache for all region server information?
Harsh J 2012-08-23, 13:35
Hi Lin,

On Thu, Aug 23, 2012 at 4:31 PM, Lin Ma <[EMAIL PROTECTED]> wrote:
> Thank you Abhishek,
>
> Two more comments,
>
> -- "Client only caches information as needed for its queries and not
> necessarily for 'all' region servers." -- how did client know which region
> server information is necessary to be cached in current HBase
> implementation?

What Abhishek meant here is that it caches only the needed table's
rows from META. It also only caches the specific region required for
the row you're looking up/operating on, AFAICT.

> -- When the client loads region server information for the first time? Did
> client persistent cache information at client side about region server
> information?

The client loads up regionserver information for a table, when it is
requested to perform an operation on that table (on a specific row or
the whole). It does not immediately, upon initialization, cache the
whole of META's contents.

Your question makes sense though, that it does seem to be such that a
client *may* use quite a bit of memory space in trying to cache the
META entries locally, but practically we've not had this cause issues
for users yet. The amount of memory cached for META far outweighs the
other items it caches (scan results, etc.). At least I have not seen
any reports of excessive client memory usage just due to region
locations of tables being cached.

I think there's more benefits storing/caching it than not doing so,
and so far we've not needed the extra complexity of persisting the
cache to a local or non-RAM storage than keeping it in memory.

--
Harsh J