Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> hadoop filesystem corrupt


+
Mohit Vadhera 2013-03-01, 18:52
Copy link to this message
-
Re: hadoop filesystem corrupt
Hi,

I solved this problem by running command " hadoop fsck / -delete" . As I
written in my email that partial data has been moved and remaining is not
moved. Can i merge the data again .  Below are directories.  How can i sure
the hadoop cluster is running fine . Is there any way to verify ?

# ll  /var/lib/cache/hdfs/hdfs/dfs/data       (running data)
total 8
drwxr-xr-x. 3 hdfs hdfs 4096 Dec 27 13:31 current
-rw-r--r--  1 hdfs hdfs   29 Mar  1 12:50 in_use.lock
Data that is not
# ll  /var/lib/cache/hdfs/hdfs/dfs/data_brk/ old data
total 4
drwx------ 3 hdfs hdfs 4096 Mar  1 12:20 current

size
# sudo du -sh  /var/lib/cache/hdfs/hdfs/dfs/*
171G    /mnt/san1/hdfs/hdfs/dfs/data
30G     /mnt/san1/hdfs/hdfs/dfs/data_brk
411M    /mnt/san1/hdfs/hdfs/dfs/name
8.0K    /mnt/san1/hdfs/hdfs/dfs/namesecondary

Thanks,
On Sat, Mar 2, 2013 at 1:02 AM, Jean-Marc Spaggiari <[EMAIL PROTECTED]
> wrote:

> Hi Mohit,
>
> Is your replication factor really setup to 1?
>
> "Default replication factor:    1"
>
> Also, can you look into you data directories and ensure you always
> have the right structur and all the related META files?
>
> JM
>
> 2013/3/1 Mohit Vadhera <[EMAIL PROTECTED]>:
> > Hi,
> >
> > While moving the data my data folder didn't move successfully. Now while
> > running the FSCK i get the file system is corrupt. Can anybody help me at
> > the earliest please. I shall be thankful
> >
> > $ sudo - hdfs hadoop fsck /
> >
> > -----------------------------
> > -------------------------
> >
> > ...Status: CORRUPT
> >  Total size:    739698399117 B
> >  Total dirs:    2179
> >  Total files:   9064 (Files currently being written: 1)
> >  Total blocks (validated):      17060 (avg. block size 43358640 B)
> >   ********************************
> >   CORRUPT FILES:        4
> >   CORRUPT BLOCKS:       4
> >   ********************************
> >  Minimally replicated blocks:   17060 (100.0 %)
> >  Over-replicated blocks:        0 (0.0 %)
> >  Under-replicated blocks:       232 (1.3599062 %)
> >  Mis-replicated blocks:         0 (0.0 %)
> >  Default replication factor:    1
> >  Average block replication:     1.0
> >  Corrupt blocks:                4
> >  Missing replicas:              2088 (10.904533 %)
> >  Number of data-nodes:          1
> >  Number of racks:               1
> > FSCK ended at Fri Mar 01 13:56:41 EST 2013 in 509 milliseconds
> >
> >
> > The filesystem under path '/' is CORRUPT
> >
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB