Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> How does BlockCache find relevant Blocks?

yun peng 2013-05-29, 11:10
Anoop John 2013-05-29, 11:45
Copy link to this message
Re: How does BlockCache find relevant Blocks?
Thanks for the clarification, and I am interested in knowing a bit deep...

So how does hbase maintains which block is in memory and which is not.
is there an extra level of index (like meta data index) here in memory? if
is, is it a hash index or other?...

On Wed, May 29, 2013 at 7:45 AM, Anoop John <[EMAIL PROTECTED]> wrote:

> There is an index for the blocks in a HFile. This index contains details
> like start row in the block, its offset and length in the HFile...   So as
> a 1st setp to get a rowkey, we will find this rk can be present in which
> HFile block.. (I am assuming only one HFile as of now)..   Now we will see
> whether this block is already present in the block cache or not. If so try
> finding the row within that block..   If this HFile block is not present in
> the cache, we will load it from HDFS..
> Hope this helps u..
> -Anoop-
> On Wed, May 29, 2013 at 4:40 PM, yun peng <[EMAIL PROTECTED]> wrote:
> > Hi, All,
> > Given a large-sized BlockCache, I am wondering how HBase performs to
> search
> > a block of a requested rowkey. Is there any index structure (in-memory)
> > inside BlockCache? or it search in a brute force way, (which seems
> unlikely
> > though :-...)
> >
> > I am aware of reference guide online, but have not yet find the answer...
> > http://hbase.apache.org/book.html#block.cache
> >
> > Thx,
> > Yun
> >
Anoop John 2013-05-29, 12:15