Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - How does BlockCache find relevant Blocks?


Copy link to this message
-
Re: How does BlockCache find relevant Blocks?
Anoop John 2013-05-29, 12:15
Block cache maintain the blocks in memory with a key, value DS.   The value
is the block and the key (BlockCacheKey) includes the HFile name and offset
of the block in this file...    As I said 1st we will determine in which
block this RK can be found. So we know the HFile name and offset now.  We
can create the BlockCacheKey and do a get(key)

-Anoop-

On Wed, May 29, 2013 at 5:37 PM, yun peng <[EMAIL PROTECTED]> wrote:

> Thanks for the clarification, and I am interested in knowing a bit deep...
>
> So how does hbase maintains which block is in memory and which is not.
> is there an extra level of index (like meta data index) here in memory? if
> there
> is, is it a hash index or other?...
>
> Regards
> Yun
>
>
> On Wed, May 29, 2013 at 7:45 AM, Anoop John <[EMAIL PROTECTED]> wrote:
>
> > There is an index for the blocks in a HFile. This index contains details
> > like start row in the block, its offset and length in the HFile...   So
> as
> > a 1st setp to get a rowkey, we will find this rk can be present in which
> > HFile block.. (I am assuming only one HFile as of now)..   Now we will
> see
> > whether this block is already present in the block cache or not. If so
> try
> > finding the row within that block..   If this HFile block is not present
> in
> > the cache, we will load it from HDFS..
> >
> > Hope this helps u..
> >
> > -Anoop-
> >
> > On Wed, May 29, 2013 at 4:40 PM, yun peng <[EMAIL PROTECTED]> wrote:
> >
> > > Hi, All,
> > > Given a large-sized BlockCache, I am wondering how HBase performs to
> > search
> > > a block of a requested rowkey. Is there any index structure (in-memory)
> > > inside BlockCache? or it search in a brute force way, (which seems
> > unlikely
> > > though :-...)
> > >
> > > I am aware of reference guide online, but have not yet find the
> answer...
> > > http://hbase.apache.org/book.html#block.cache
> > >
> > > Thx,
> > > Yun
> > >
> >
>