Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> RegionServer shutdown with ScanWildcardColumnTracker exception


Copy link to this message
-
RegionServer shutdown with ScanWildcardColumnTracker exception
Hi all,

I had a regionserver crushed during counters increment. Looking at the
regionserver log I saw:

org.apache.hadoop.hbase.DroppedSnapshotException: region: TABLE_NAME,
ROW_KEY...at
org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1472)
        at
org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1351)
        at
org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:1292)
        at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:406)
        at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:380)
        at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.run(MemStoreFlusher.java:243)
        at java.lang.Thread.run(Thread.java:722)
Caused by: java.io.IOException: ScanWildcardColumnTracker.checkColumn ran
into a column actually smaller than the previous column: *QUALIFIER*
at
org.apache.hadoop.hbase.regionserver.ScanWildcardColumnTracker.checkColumn(ScanWildcardColumnTracker.java:104)
        at
org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:354)
        at
org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:362)
        at
org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:311)
        at
org.apache.hadoop.hbase.regionserver.Store.internalFlushCache(Store.java:738)
        at
org.apache.hadoop.hbase.regionserver.Store.flushCache(Store.java:673)
        at
org.apache.hadoop.hbase.regionserver.Store.access$400(Store.java:108)
        at
org.apache.hadoop.hbase.regionserver.Store$StoreFlusherImpl.flushCache(Store.java:2276)
        at
org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1447)

The strange thing is that the *QUALIFER* name as it appears in the log is
misspelled.... there is no, and never was such qualifier name.

Thanks,

Amit.
+
ramkrishna vasudevan 2013-04-17, 08:47
+
Amit Sela 2013-04-17, 09:06
+
ramkrishna vasudevan 2013-04-17, 10:42
+
Amit Sela 2013-04-17, 11:47
+
ramkrishna vasudevan 2013-04-17, 15:56
+
Amit Sela 2013-04-17, 16:10
+
ramkrishna vasudevan 2013-04-17, 16:17
+
ramkrishna vasudevan 2013-04-17, 17:18
+
Amit Sela 2013-04-17, 18:34
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB