Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> How does BlockCache find relevant Blocks?


+
yun peng 2013-05-29, 11:10
+
Anoop John 2013-05-29, 11:45
+
yun peng 2013-05-29, 12:07
Copy link to this message
-
Re: How does BlockCache find relevant Blocks?
Block cache maintain the blocks in memory with a key, value DS.   The value
is the block and the key (BlockCacheKey) includes the HFile name and offset
of the block in this file...    As I said 1st we will determine in which
block this RK can be found. So we know the HFile name and offset now.  We
can create the BlockCacheKey and do a get(key)

-Anoop-

On Wed, May 29, 2013 at 5:37 PM, yun peng <[EMAIL PROTECTED]> wrote:

> Thanks for the clarification, and I am interested in knowing a bit deep...
>
> So how does hbase maintains which block is in memory and which is not.
> is there an extra level of index (like meta data index) here in memory? if
> there
> is, is it a hash index or other?...
>
> Regards
> Yun
>
>
> On Wed, May 29, 2013 at 7:45 AM, Anoop John <[EMAIL PROTECTED]> wrote:
>
> > There is an index for the blocks in a HFile. This index contains details
> > like start row in the block, its offset and length in the HFile...   So
> as
> > a 1st setp to get a rowkey, we will find this rk can be present in which
> > HFile block.. (I am assuming only one HFile as of now)..   Now we will
> see
> > whether this block is already present in the block cache or not. If so
> try
> > finding the row within that block..   If this HFile block is not present
> in
> > the cache, we will load it from HDFS..
> >
> > Hope this helps u..
> >
> > -Anoop-
> >
> > On Wed, May 29, 2013 at 4:40 PM, yun peng <[EMAIL PROTECTED]> wrote:
> >
> > > Hi, All,
> > > Given a large-sized BlockCache, I am wondering how HBase performs to
> > search
> > > a block of a requested rowkey. Is there any index structure (in-memory)
> > > inside BlockCache? or it search in a brute force way, (which seems
> > unlikely
> > > though :-...)
> > >
> > > I am aware of reference guide online, but have not yet find the
> answer...
> > > http://hbase.apache.org/book.html#block.cache
> > >
> > > Thx,
> > > Yun
> > >
> >
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB