Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> optimizing block cache requests + eviction


+
Viral Bajaria 2013-07-08, 10:04
+
Ted Yu 2013-07-08, 13:48
+
Andrew Purtell 2013-07-08, 15:49
+
Viral Bajaria 2013-07-08, 19:22
+
Andrew Purtell 2013-07-08, 22:53
+
Viral Bajaria 2013-07-08, 23:28
+
Jean-Daniel Cryans 2013-07-08, 23:37
+
Viral Bajaria 2013-07-08, 23:45
Copy link to this message
-
Re: optimizing block cache requests + eviction
FYI, if u disable your block cache - you will ask for "Index" blocks for
every single request. So such a high rate of request is plausible for Index
blocks even when your requests are totally random on your data.

Varun
On Mon, Jul 8, 2013 at 4:45 PM, Viral Bajaria <[EMAIL PROTECTED]>wrote:

> Good question. When I looked at the logs, it's not clear from it whether
> it's reading a meta or data block. Is there any kind of log line that
> indicates that ? Given that it's saying that it's ready from a startOffset
> I would assume this is a data block.
>
> A question that comes to mind, is this read doing a seek to that position
> directly or is it going to cache the block ? Looks like it is not caching
> the block if it's reading directly from a given offset. Or am I wrong ?
>
> Following is a sample line that I used while debugging:
> 2013-07-08 22:58:55,221 DEBUG org.apache.hadoop.hdfs.DFSClient: New
> BlockReaderLocal for file
> /mnt/data/current/subdir34/subdir26/blk_-448970697931783518 of size
> 67108864 startOffset 13006577 length 54102287 short circuit checksum true
>
> On Mon, Jul 8, 2013 at 4:37 PM, Jean-Daniel Cryans <[EMAIL PROTECTED]
> >wrote:
>
> > Do you know if it's a data or meta block?
>
+
Viral Bajaria 2013-07-09, 00:20
+
Jean-Daniel Cryans 2013-07-08, 23:51
+
Vladimir Rodionov 2013-07-08, 17:45