Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop, mail # user - Re: hadoop filesystem corrupt


Copy link to this message
-
Re: hadoop filesystem corrupt
Mohit Vadhera 2013-03-03, 10:47
Can anybody let me know how to get block size back into the running system
? I want data_brk files back into the directory data ? I want the running
filesystem should start it reading .

$ sudo ls -l  /mnt/san1/hdfs/hdfs/dfs/
total 16
drwx------. 3 hdfs hdfs 4096 Mar  1 12:50 data
drwx------  3 hdfs hdfs 4096 Mar  1 12:20 data_brk
drwxr-xr-x  3 hdfs hdfs 4096 Mar  1 12:50 name
drwxr-xr-x  3 hdfs hdfs 4096 Mar  1 13:04 namesecondary
Thanks,

On Sat, Mar 2, 2013 at 2:31 AM, Mohit Vadhera
<[EMAIL PROTECTED]>wrote:

> Hi,
>
> I solved this problem by running command " hadoop fsck / -delete" . As I
> written in my email that partial data has been moved and remaining is not
> moved. Can i merge the data again .  Below are directories.  How can i sure
> the hadoop cluster is running fine . Is there any way to verify ?
>
> # ll  /var/lib/cache/hdfs/hdfs/dfs/data       (running data)
> total 8
> drwxr-xr-x. 3 hdfs hdfs 4096 Dec 27 13:31 current
> -rw-r--r--  1 hdfs hdfs   29 Mar  1 12:50 in_use.lock
>
>
> Data that is not
> # ll  /var/lib/cache/hdfs/hdfs/dfs/data_brk/ old data
> total 4
> drwx------ 3 hdfs hdfs 4096 Mar  1 12:20 current
>
> size
> # sudo du -sh  /var/lib/cache/hdfs/hdfs/dfs/*
> 171G    /mnt/san1/hdfs/hdfs/dfs/data
> 30G     /mnt/san1/hdfs/hdfs/dfs/data_brk
> 411M    /mnt/san1/hdfs/hdfs/dfs/name
> 8.0K    /mnt/san1/hdfs/hdfs/dfs/namesecondary
>
> Thanks,
>
>
>
>
> On Sat, Mar 2, 2013 at 1:02 AM, Jean-Marc Spaggiari <
> [EMAIL PROTECTED]> wrote:
>
>> Hi Mohit,
>>
>> Is your replication factor really setup to 1?
>>
>> "Default replication factor:    1"
>>
>> Also, can you look into you data directories and ensure you always
>> have the right structur and all the related META files?
>>
>> JM
>>
>> 2013/3/1 Mohit Vadhera <[EMAIL PROTECTED]>:
>> > Hi,
>> >
>> > While moving the data my data folder didn't move successfully. Now while
>> > running the FSCK i get the file system is corrupt. Can anybody help me
>> at
>> > the earliest please. I shall be thankful
>> >
>> > $ sudo - hdfs hadoop fsck /
>> >
>> > -----------------------------
>> > -------------------------
>> >
>> > ...Status: CORRUPT
>> >  Total size:    739698399117 B
>> >  Total dirs:    2179
>> >  Total files:   9064 (Files currently being written: 1)
>> >  Total blocks (validated):      17060 (avg. block size 43358640 B)
>> >   ********************************
>> >   CORRUPT FILES:        4
>> >   CORRUPT BLOCKS:       4
>> >   ********************************
>> >  Minimally replicated blocks:   17060 (100.0 %)
>> >  Over-replicated blocks:        0 (0.0 %)
>> >  Under-replicated blocks:       232 (1.3599062 %)
>> >  Mis-replicated blocks:         0 (0.0 %)
>> >  Default replication factor:    1
>> >  Average block replication:     1.0
>> >  Corrupt blocks:                4
>> >  Missing replicas:              2088 (10.904533 %)
>> >  Number of data-nodes:          1
>> >  Number of racks:               1
>> > FSCK ended at Fri Mar 01 13:56:41 EST 2013 in 509 milliseconds
>> >
>> >
>> > The filesystem under path '/' is CORRUPT
>> >
>>
>
>