Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Re: hadoop filesystem corrupt


Copy link to this message
-
Re: hadoop filesystem corrupt
Can anybody let me know how to get block size back into the running system
? I want data_brk files back into the directory data ? I want the running
filesystem should start it reading .

$ sudo ls -l  /mnt/san1/hdfs/hdfs/dfs/
total 16
drwx------. 3 hdfs hdfs 4096 Mar  1 12:50 data
drwx------  3 hdfs hdfs 4096 Mar  1 12:20 data_brk
drwxr-xr-x  3 hdfs hdfs 4096 Mar  1 12:50 name
drwxr-xr-x  3 hdfs hdfs 4096 Mar  1 13:04 namesecondary
Thanks,

On Sat, Mar 2, 2013 at 2:31 AM, Mohit Vadhera
<[EMAIL PROTECTED]>wrote:

> Hi,
>
> I solved this problem by running command " hadoop fsck / -delete" . As I
> written in my email that partial data has been moved and remaining is not
> moved. Can i merge the data again .  Below are directories.  How can i sure
> the hadoop cluster is running fine . Is there any way to verify ?
>
> # ll  /var/lib/cache/hdfs/hdfs/dfs/data       (running data)
> total 8
> drwxr-xr-x. 3 hdfs hdfs 4096 Dec 27 13:31 current
> -rw-r--r--  1 hdfs hdfs   29 Mar  1 12:50 in_use.lock
>
>
> Data that is not
> # ll  /var/lib/cache/hdfs/hdfs/dfs/data_brk/ old data
> total 4
> drwx------ 3 hdfs hdfs 4096 Mar  1 12:20 current
>
> size
> # sudo du -sh  /var/lib/cache/hdfs/hdfs/dfs/*
> 171G    /mnt/san1/hdfs/hdfs/dfs/data
> 30G     /mnt/san1/hdfs/hdfs/dfs/data_brk
> 411M    /mnt/san1/hdfs/hdfs/dfs/name
> 8.0K    /mnt/san1/hdfs/hdfs/dfs/namesecondary
>
> Thanks,
>
>
>
>
> On Sat, Mar 2, 2013 at 1:02 AM, Jean-Marc Spaggiari <
> [EMAIL PROTECTED]> wrote:
>
>> Hi Mohit,
>>
>> Is your replication factor really setup to 1?
>>
>> "Default replication factor:    1"
>>
>> Also, can you look into you data directories and ensure you always
>> have the right structur and all the related META files?
>>
>> JM
>>
>> 2013/3/1 Mohit Vadhera <[EMAIL PROTECTED]>:
>> > Hi,
>> >
>> > While moving the data my data folder didn't move successfully. Now while
>> > running the FSCK i get the file system is corrupt. Can anybody help me
>> at
>> > the earliest please. I shall be thankful
>> >
>> > $ sudo - hdfs hadoop fsck /
>> >
>> > -----------------------------
>> > -------------------------
>> >
>> > ...Status: CORRUPT
>> >  Total size:    739698399117 B
>> >  Total dirs:    2179
>> >  Total files:   9064 (Files currently being written: 1)
>> >  Total blocks (validated):      17060 (avg. block size 43358640 B)
>> >   ********************************
>> >   CORRUPT FILES:        4
>> >   CORRUPT BLOCKS:       4
>> >   ********************************
>> >  Minimally replicated blocks:   17060 (100.0 %)
>> >  Over-replicated blocks:        0 (0.0 %)
>> >  Under-replicated blocks:       232 (1.3599062 %)
>> >  Mis-replicated blocks:         0 (0.0 %)
>> >  Default replication factor:    1
>> >  Average block replication:     1.0
>> >  Corrupt blocks:                4
>> >  Missing replicas:              2088 (10.904533 %)
>> >  Number of data-nodes:          1
>> >  Number of racks:               1
>> > FSCK ended at Fri Mar 01 13:56:41 EST 2013 in 509 milliseconds
>> >
>> >
>> > The filesystem under path '/' is CORRUPT
>> >
>>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB