Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # user - LRU stats


+
Varun Sharma 2013-01-27, 03:05
+
Ted Yu 2013-01-27, 05:24
+
Varun Sharma 2013-01-27, 09:04
Copy link to this message
-
Re: LRU stats
Jean-Daniel Cryans 2013-01-28, 19:27
IIRC when a file closes it will evict its own blocks since they won't
be used after that.

J-D

On Sun, Jan 27, 2013 at 1:04 AM, Varun Sharma <[EMAIL PROTECTED]> wrote:
> Since i am using only 10 % of allocated cache, I think EvictionThread never
> ran - hence, I see the value 0. What's mysterious is who is calling
> evictBlock then, because there are non zero evicted blocks ?
>
> On Sat, Jan 26, 2013 at 9:24 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
>> Evicted block count stat is gathered in:
>>
>>   protected long evictBlock(CachedBlock block) {
>>
>>     map.remove(block.getCacheKey());
>>
>>     updateSizeMetrics(block, true);
>>
>>     elements.decrementAndGet();
>>
>>     stats.evicted();
>>
>> In EvictionThread#run():
>>
>>         LruBlockCache cache = this.cache.get();
>>
>>         if(cache == null) break;
>>
>>         cache.evict();
>>
>> where cache is declared:
>>
>>     private WeakReference<LruBlockCache> cache;
>>
>> Looks like cache became null (due to heap pressure), leading to 0 evictions
>> that you saw.
>>
>>
>> Cheers
>>
>>
>> On Sat, Jan 26, 2013 at 7:05 PM, Varun Sharma <[EMAIL PROTECTED]> wrote:
>>
>> > Hi,
>> >
>> > I am running hbase 0.94.3 and I am trying to understand the LRU cache
>> stats
>> > here:
>> >
>> > LRU Stats: total=428.75 MB, free=4.05 GB, max=4.46 GB, blocks=35961,
>> > accesses=23808283, hits=23098837, hitRatio=97.02%,
>> > cachingAccesses=23135166, cachingHits=23066278,
>> > cachingHitsRatio=99.70%, *evictions=0,
>> > evicted=33211*
>> > *
>> > *
>> > I don't understanding why the number of evictions is 0 - which means that
>> > the number of eviction thread runs is 0, AFAIK. Also, I see that 30K
>> blocks
>> > were evicted - how is that possible ? I have set the hfile evict on close
>> > to the default value - which is false - so blocks should not be evicted
>> > when hfiles are closed. Where are the evicted blocks coming from when the
>> > block is mostly empty and no eviction runs have occured.
>> >
>> > Thanks
>> > Varun
>> >
>>