Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> HDFS inconsistency issue


Copy link to this message
-
HDFS inconsistency issue
We are hitting a weird HDFS issue after a good number of Hadoop nodes
simultaneously crashed.

The problem is that after all the down servers came back, the HDFS complained
there was 1 missing block.
But the file that that block belongs to has already been deleted after the
crash. So it's an orphan block.
Because the block doesn't belong to any file, there is no way to delete that
block. FSCK also failed with a

cause:java.io.IOException: Premature EOF exception.

A side effect now is that HDFS won't free up any space even after Trash bin was
emptied. The space utilization

just constantly went up.

Any suggestion on how to resolve this issue is highly appreciated!

Thanks,
-Songting