Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> Meta vs Stores / first and last keys and corrupted HFiles.


Copy link to this message
-
Re: Meta vs Stores / first and last keys and corrupted HFiles.
Um, I may be missing something here, but is it really an HFile?
Looks like an hlog to me.

org.apache.hadoop.hbase.io.
hfile.CorruptHFileException: Problem reading
HFile Trailer from file
hdfs://node3:9000/hbase/work_proposed/db83e64f34a5a608335818321f1a6c32/.oldlogs/hlog.1377344531526
On Mon, Aug 26, 2013 at 12:31 PM, Jean-Marc Spaggiari <
[EMAIL PROTECTED]> wrote:

> Hi,
>
> Don't ask me how, but I have one table in a pretty strange state.
>
> First, seems that I have one corrupted HFile (at least).
>
> FirstKey return null. (Same for StopKey since header is corrupted).
>
> Exception in thread "main" java.lang.NullPointerException
>     at org.apache.hadoop.hbase.KeyValue.keyToString(KeyValue.java:716)
>     at
>
> org.apache.hadoop.hbase.io.hfile.AbstractHFileReader.toStringFirstKey(AbstractHFileReader.java:138)
>     at
>
> org.apache.hadoop.hbase.io.hfile.AbstractHFileReader.toString(AbstractHFileReader.java:149)
>     at
>
> org.apache.hadoop.hbase.io.hfile.HFilePrettyPrinter.printMeta(HFilePrettyPrinter.java:325)
>     at
>
> org.apache.hadoop.hbase.io.hfile.HFilePrettyPrinter.processFile(HFilePrettyPrinter.java:241)
>     at
>
> org.apache.hadoop.hbase.io.hfile.HFilePrettyPrinter.run(HFilePrettyPrinter.java:196)
>     at org.apache.hadoop.hbase.io.hfile.HFile.main(HFile.java:756)
>
> Because the header is not correct.
>
> org.apache.hadoop.hbase.io.hfile.CorruptHFileException: Problem reading
> HFile Trailer from file
>
> hdfs://node3:9000/hbase/work_proposed/db83e64f34a5a608335818321f1a6c32/.oldlogs/hlog.1377344531526
>     at
> org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:551)
>     at
>
> org.apache.hadoop.hbase.io.hfile.HFile.createReaderWithEncoding(HFile.java:595)
>     at org.apache.hadoop.hbase.io.hfile.HFile.createReader(HFile.java:642)
>     at
>
> org.apache.hadoop.hbase.io.hfile.HFilePrettyPrinter.processFile(HFilePrettyPrinter.java:217)
>     at
>
> org.apache.hadoop.hbase.io.hfile.HFilePrettyPrinter.run(HFilePrettyPrinter.java:196)
>     at org.apache.hadoop.hbase.io.hfile.HFile.main(HFile.java:756)
> Caused by: java.lang.IllegalArgumentException: Invalid HFile version:
> 16275367 (expected to be between 1 and 2)
>     at
> org.apache.hadoop.hbase.io.hfile.HFile.checkFormatVersion(HFile.java:771)
>     at
>
> org.apache.hadoop.hbase.io.hfile.FixedFileTrailer.readFromStream(FixedFileTrailer.java:323)
>     at
> org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:549)
>     ... 5 more
>
> HBCK don't detect that:
> 0 inconsistencies detected.
> Status: OK
>
> Also, I have an issue with the first key / last key reported by the stores
> overlapping when the one reported by the META are not.
>
> Store:
>     firstKey=\xF5\x9A\xEA&\x00\x00\x00\x00...
>     lastKey=\xFF\xFF\xFF\xFE\x00\x00\x00\x00...
> Meta:
>     firstKey=\xF5\x9A\xEA&\x00\x00\x00\x00
>     lastKey=\xF5\x9B@}\x00\x00\x00\x00...
>
>
> So, few things.
>
> 1) We should add something into HBCK to check the HFile format again
> corruption.
> 2) We should add something into HBCK to validate META regions boundaries
> against Store Files
> 3) How can I repaire my HFile? ;)
>
> I'm already working on #2 and will have something ready soon. Then I will
> most probably more to #1. But I only have detection dont for now. I'm not
> sure exactly what are the correct steps to repair...
>
> JM
>

--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.