Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - optimizing block cache requests + eviction


Copy link to this message
-
Re: optimizing block cache requests + eviction
Jean-Daniel Cryans 2013-07-08, 23:51
meta blocks are at the end:
http://hbase.apache.org/book.html#d2617e12979, a way to tell would be
by logging from the HBase side but then I guess it's hard to reconcile
with which file we're actually reading from...

Regarding your second question, you are asking if we block HDFS
blocks? We don't, since we don't even know about HDFS blocks. The
BlockReader seeks into the file and return whatever data is asked.

J-D

On Mon, Jul 8, 2013 at 4:45 PM, Viral Bajaria <[EMAIL PROTECTED]> wrote:
> Good question. When I looked at the logs, it's not clear from it whether
> it's reading a meta or data block. Is there any kind of log line that
> indicates that ? Given that it's saying that it's ready from a startOffset
> I would assume this is a data block.
>
> A question that comes to mind, is this read doing a seek to that position
> directly or is it going to cache the block ? Looks like it is not caching
> the block if it's reading directly from a given offset. Or am I wrong ?
>
> Following is a sample line that I used while debugging:
> 2013-07-08 22:58:55,221 DEBUG org.apache.hadoop.hdfs.DFSClient: New
> BlockReaderLocal for file
> /mnt/data/current/subdir34/subdir26/blk_-448970697931783518 of size
> 67108864 startOffset 13006577 length 54102287 short circuit checksum true
>
> On Mon, Jul 8, 2013 at 4:37 PM, Jean-Daniel Cryans <[EMAIL PROTECTED]>wrote:
>
>> Do you know if it's a data or meta block?