Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Off-heap block cache fails in 0.94.6


Copy link to this message
-
Re: Off-heap block cache fails in 0.94.6
Could you deploy a fix if we made one for you Dean?
St.Ack
On Tue, Jan 21, 2014 at 11:51 AM, Dean <[EMAIL PROTECTED]> wrote:

> Hi,
>
> We recently upgraded our production cluster from 0.92.1 to 0.94.6 (CDH
> 4.5.0).  While running 0.92.1, we used the "experimental" off-heap block
> cache with good results, but it appears broken and unusable in 0.94.6  We
> seem to have bumped into HBASE-6048 or HBASE-7136, though we hit the error
> with increments rather than scans.
>
> Both Is it just that nobody uses off-heap cache, or is there a particular
> RS configuration or table schema that causes issues?  We found that the
> off-heap cache was really useful in allowing us to cache the blocks we
> needed without resorting to large heap sizes or trashing the OS filesystem
> cache during scans.
>
> Cheers,
>
> Dean
>
> The stack trace is:
> Mon Jan 20 21:31:37 GMT 2014, org.apache.hadoop.hbase.
> client.HTable$7@37e687d1, java.io.IOException: java.io.IOException:
> java.lang.IllegalStateException: Schema metrics requested before table/CF
> name initialization: {"tableName":"null","cfName":"null"}
>         at org.apache.hadoop.hbase.regionserver.metrics.SchemaConfigured.
> getSchemaMetrics(SchemaConfigured.java:180)
>         at org.apache.hadoop.hbase.io.hfile.LruBlockCache.
> updateSizeMetrics(LruBlockCache.java:337)
>         at org.apache.hadoop.hbase.io.hfile.LruBlockCache.
> cacheBlock(LruBlockCache.java:292)
>         at org.apache.hadoop.hbase.io.hfile.LruBlockCache.
> cacheBlock(LruBlockCache.java:320)
>         at org.apache.hadoop.hbase.io.hfile.DoubleBlockCache.
> getBlock(DoubleBlockCache.java:102)
>         at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(
> HFileReaderV2.java:303)
>         at org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$
> BlockIndexReader.loadDataBlockWithScanInfo(HFileBlockIndex.java:254)
>         at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$
> AbstractScannerV2.seekTo(HFileReaderV2.java:480)
>         at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$
> AbstractScannerV2.reseekTo(HFileReaderV2.java:530)
>         at org.apache.hadoop.hbase.regionserver.StoreFileScanner.
> reseekAtOrAfter(StoreFileScanner.java:236)
>         at org.apache.hadoop.hbase.regionserver.StoreFileScanner.
> reseek(StoreFileScanner.java:161)
>         at org.apache.hadoop.hbase.regionserver.StoreFileScanner.
> enforceSeek(StoreFileScanner.java:349)
>         at org.apache.hadoop.hbase.regionserver.KeyValueHeap.
> pollRealKV(KeyValueHeap.java:355)
>         at org.apache.hadoop.hbase.regionserver.KeyValueHeap.
> generalizedSeek(KeyValueHeap.java:312)
>         at org.apache.hadoop.hbase.regionserver.KeyValueHeap.
> requestSeek(KeyValueHeap.java:277)
>         at org.apache.hadoop.hbase.regionserver.StoreScanner.
> reseek(StoreScanner.java:543)
>         at org.apache.hadoop.hbase.regionserver.StoreScanner.
> next(StoreScanner.java:411)
>         at org.apache.hadoop.hbase.regionserver.KeyValueHeap.
> next(KeyValueHeap.java:143)
>         at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.
> populateResult(HRegion.java:3867)
>         at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.
> nextInternal(HRegion.java:3939)
>         at org.apache.hadoop.hbase.regionserver.HRegion$
> RegionScannerImpl.nextRaw(HRegion.java:3810)
>         at org.apache.hadoop.hbase.regionserver.HRegion$
> RegionScannerImpl.next(HRegion.java:3791)
>         at org.apache.hadoop.hbase.regionserver.HRegion$
> RegionScannerImpl.next(HRegion.java:3834)
>         at org.apache.hadoop.hbase.regionserver.HRegion.get(
> HRegion.java:4760)
>         at org.apache.hadoop.hbase.regionserver.HRegion.
> increment(HRegion.java:5202)
>         at org.apache.hadoop.hbase.regionserver.HRegionServer.
> increment(HRegionServer.java:3532)
>         at sun.reflect.GeneratedMethodAccessor32.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB