Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> HDFS is corrupt, need to salvage the data.

Copy link to this message
Re: HDFS is corrupt, need to salvage the data.

How many Datanodes do you have.
>From the output it looks like at the point when you ran fsck, you had only one datanode connected to your NameNode. Did you have others?
Also, I see that your default replication is set to 1. Can you check if  your datanodes are up and running.

----- Original Message ----
From: Mayuran Yogarajah <[EMAIL PROTECTED]>
Sent: Monday, March 9, 2009 5:20:37 PM
Subject: HDFS is corrupt, need to salvage the data.

Hello, it seems the HDFS in my cluster is corrupt.  This is the output from hadoop fsck:
Total size:    9196815693 B
Total dirs:    17
Total files:   157
Total blocks:  157 (avg. block size 58578443 B)
CORRUPT FILES:        157
MISSING SIZE:         9196815693 B
Minimally replicated blocks:   0 (0.0 %)
Over-replicated blocks:        0 (0.0 %)
Under-replicated blocks:       0 (0.0 %)
Mis-replicated blocks:         0 (0.0 %)
Default replication factor:    1
Average block replication:     0.0
Missing replicas:              0
Number of data-nodes:          1
Number of racks:               1

It seems to say that there is 1 block missing from every file that was in the cluster..

I'm not sure how to proceed so any guidance would be much appreciated.  My primary
concern is recovering the data.