Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> HLogSplit error with hadoop-2.0.3-alpha and hbase trunk


Copy link to this message
-
Re: HLogSplit error with hadoop-2.0.3-alpha and hbase trunk
Ok so i tried this out with hadoop 2.0.4 and also with Sergey's patch.  The
issue is reproducible in all version of hadoop but not always.
I am able to get the errors like this

2013-05-07 17:11:08,827 WARN
[SplitLogWorker-ram.sh.intel.com,60020,1367961009182]
org.apache.hadoop.hbase.regionserver.SplitLogWorker: log splitting of .logs/
ram.sh.intel.com,60020,1367960957620-splitting/ram.sh.intel.com%2C60020%2C1367960957620.1367960993389
failed, returning error
java.io.IOException: Error  while reading 1 WAL KVs; started reading at
589822 and read up to 589824
at
org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader.readNext(ProtobufLogReader.java:162)
at
org.apache.hadoop.hbase.regionserver.wal.ReaderBase.next(ReaderBase.java:88)
at
org.apache.hadoop.hbase.regionserver.wal.ReaderBase.next(ReaderBase.java:75)
at
org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.getNextLogLine(HLogSplitter.java:775)
at
org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.splitLogFile(HLogSplitter.java:459)
at
org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.splitLogFile(HLogSplitter.java:388)
at
org.apache.hadoop.hbase.regionserver.SplitLogWorker$1.exec(SplitLogWorker.java:115)
at
org.apache.hadoop.hbase.regionserver.SplitLogWorker.grabTask(SplitLogWorker.java:278)
at
org.apache.hadoop.hbase.regionserver.SplitLogWorker.taskLoop(SplitLogWorker.java:199)
at
org.apache.hadoop.hbase.regionserver.SplitLogWorker.run(SplitLogWorker.java:166)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.IOException: Failed read of int length 2
at org.apache.hadoop.hbase.KeyValue.iscreate(KeyValue.java:2335)
at
org.apache.hadoop.hbase.codec.KeyValueCodec$KeyValueDecoder.parseCell(KeyValueCodec.java:66)
at org.apache.hadoop.hbase.codec.BaseDecoder.advance(BaseDecoder.java:46)
at
org.apache.hadoop.hbase.regionserver.wal.WALEdit.readFromCells(WALEdit.java:199)
at
org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader.readNext(ProtobufLogReader.java:143)
... 10 more
and sometimes
java.io.IOException: Failed read of int length 1
at org.apache.hadoop.hbase.KeyValue.iscreate(KeyValue.java:2335)
at
org.apache.hadoop.hbase.codec.KeyValueCodec$KeyValueDecoder.parseCell(KeyValueCodec.java:66)
at org.apache.hadoop.hbase.codec.BaseDecoder.advance(BaseDecoder.java:41)
at
org.apache.hadoop.hbase.regionserver.wal.WALEdit.readFromCells(WALEdit.java:199)
at
org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader.readNext(ProtobufLogReader.java:137)
at
org.apache.hadoop.hbase.regionserver.wal.ReaderBase.next(ReaderBase.java:88)
at
org.apache.hadoop.hbase.regionserver.wal.ReaderBase.next(ReaderBase.java:75)
at
org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEdits(HRegion.java:2837)
at
org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEditsIfAny(HRegion.java:2755)
at
org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionStores(HRegion.java:664)
at
org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:569)
at org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:540)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4095)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4066)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4016)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:3967)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:448)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:136)
at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:130)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
I will work on this today and find out the root cause of it.

Regards
Ram
On Tue, May 7, 2013 at 8:12 AM, ramkrishna vasudevan <
[EMAIL PROTECTED]> wrote: