We are hitting a weird HDFS issue after a good number of Hadoop nodes
The problem is that after all the down servers came back, the HDFS complained
there was 1 missing block.
But the file that that block belongs to has already been deleted after the
crash. So it's an orphan block.
Because the block doesn't belong to any file, there is no way to delete that
block. FSCK also failed with a
cause:java.io.IOException: Premature EOF exception.
A side effect now is that HDFS won't free up any space even after Trash bin was
emptied. The space utilization
just constantly went up.
Any suggestion on how to resolve this issue is highly appreciated!