Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> compaction failure


Copy link to this message
-
compaction failure
Hi, :
  I found lots of error saying compaction  failed, does anybody know how to get around this
problem ?
Jimmy.

2010-06-23 00:01:17,407 DEBUG org.apache.hadoop.hbase.regionserver.CompactSplitT
hread: Compaction requested for region Spam_MsgEventTable,2010-06-22 13:17:04\x0
915e45cc95fa9201b366dc7316dd88438,1277244197913/1155282109 because: regionserver
/10.110.8.91:60020.cacheFlusher
2010-06-23 00:01:17,736 WARN org.apache.hadoop.hdfs.DFSClient: DFS Read: java.io
.IOException: Could not find target position 8305
        at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockAt(DFSClien
t.java:1556)
        at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.blockSeekTo(DFSClient
.java:1666)
        at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:1
780)
        at java.io.DataInputStream.read(DataInputStream.java:132)
        at org.apache.hadoop.hbase.io.hfile.BoundedRangeFileInputStream.read(Bou
ndedRangeFileInputStream.java:105)
        at org.apache.hadoop.io.compress.DecompressorStream.getCompressedData(De
compressorStream.java:96)
        at org.apache.hadoop.io.compress.DecompressorStream.decompress(Decompres
sorStream.java:86)
        at org.apache.hadoop.io.compress.DecompressorStream.read(DecompressorStr
eam.java:74)
        at java.io.BufferedInputStream.read1(BufferedInputStream.java:256)
        at java.io.BufferedInputStream.read(BufferedInputStream.java:317)
        at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:100)
        at org.apache.hadoop.hbase.io.hfile.HFile$Reader.decompress(HFile.java:1
018)
        at org.apache.hadoop.hbase.io.hfile.HFile$Reader.readBlock(HFile.java:96
6)
        at org.apache.hadoop.hbase.io.hfile.HFile$Reader$Scanner.next(HFile.java
:1159)
        at org.apache.hadoop.hbase.io.HalfHFileReader$1.next(HalfHFileReader.jav
a:120)
        at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileS
canner.java:58)
        at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.j
ava:79)
        at org.apache.hadoop.hbase.regionserver.MinorCompactingStoreScanner.next
(MinorCompactingStoreScanner.java:96)
        at org.apache.hadoop.hbase.regionserver.Store.compact(Store.java:920)
        at org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.ja
va:832)
        at org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.ja
va:785)
        at org.apache.hadoop.hbase.regionserver.CompactSplitThread.run(CompactSp
litThread.java:93)

2010-06-23 00:01:17,820 ERROR org.apache.hadoop.hbase.regionserver.CompactSplitT
hread: Compaction failed for region Spam_MsgEventTable,2010-06-22 13:17:04\x0915
e45cc95fa9201b366dc7316dd88438,1277244197913
java.lang.RuntimeException: java.io.IOException: Could not find target position
8305
        at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileS
canner.java:61)
        at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.j
ava:79)
        at org.apache.hadoop.hbase.regionserver.MinorCompactingStoreScanner.next
(MinorCompactingStoreScanner.java:96)
        at org.apache.hadoop.hbase.regionserver.Store.compact(Store.java:920)
        at org.apache.hadoop.hbase.regionserver.Store.compact(Store.java:764)
        at org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.ja
va:832)
        at org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.ja
va:785)
        at org.apache.hadoop.hbase.regionserver.CompactSplitThread.run(CompactSp
litThread.java:93)
Caused by: java.io.IOException: Could not find target position 8305
        at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockAt(DFSClien
t.java:1556)
        at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.blockSeekTo(DFSClient
.java:1666)
        at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:1
780)
        at java.io.DataInputStream.read(DataInputStream.java:132)
        at org.apache.hadoop.hbase.io.hfile.BoundedRangeFileInputStream.read(Bou
ndedRangeFileInputStream.java:105)
        at org.apache.hadoop.io.compress.DecompressorStream.getCompressedData(De
compressorStream.java:96)
        at org.apache.hadoop.io.compress.DecompressorStream.decompress(Decompres
sorStream.java:86)
        at org.apache.hadoop.io.compress.DecompressorStream.read(DecompressorStr
eam.java:74)
        at java.io.BufferedInputStream.read1(BufferedInputStream.java:256)
        at java.io.BufferedInputStream.read(BufferedInputStream.java:317)
        at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:100)
        at org.apache.hadoop.hbase.io.hfile.HFile$Reader.decompress(HFile.java:1
018)
        at org.apache.hadoop.hbase.io.hfile.HFile$Reader$Scanner.next(HFile.java
        at org.apache.hadoop.hbase.io.HalfHFileReader$1.next(HalfHFileReader.jav
a:120)
        at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileS
canner.java:58)
        ... 7 more
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB