Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> issue about corrupt block test


Copy link to this message
-
Re: issue about corrupt block test
Block files are not stored in a flat directory (to avoid FS limits of
max files under a dir). Instead of looking for them right under
finalized, issue a "find" query with the pattern instead and you
should be able to spot it.

On Wed, Dec 11, 2013 at 9:10 AM, ch huang <[EMAIL PROTECTED]> wrote:
> hi,maillist:
>              i try to corrupt a block of a file in my benchmark environment,
> as the following command i find blk_2504407693800874616_106252 ,it's replica
> on 192.168.10.224 is my target ,but i find all the datadir in 192.168.10.224
> ,can not fine the datafile belongs to this replic ,why?
>
> # ls
> /data/dataspace/1/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/blk_3717620888497075523_106232*
> ls: cannot access
> /data/dataspace/1/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/blk_3717620888497075523_106232*:
> No such file or directory
> [root@CHBM224 conf]# ls
> /data/dataspace/1/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/blk_2504407693800874616_106252*
> ls: cannot access
> /data/dataspace/1/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/blk_2504407693800874616_106252*:
> No such file or directory
> [root@CHBM224 conf]# ls
> /data/dataspace/2/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/blk_2504407693800874616_106252*
> ls: cannot access
> /data/dataspace/2/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/blk_2504407693800874616_106252*:
> No such file or directory
> [root@CHBM224 conf]# ls
> /data/dataspace/3/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/blk_2504407693800874616_106252*
> ls: cannot access
> /data/dataspace/3/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/blk_2504407693800874616_106252*:
> No such file or directory
> [root@CHBM224 conf]# hdfs fsck /alex/terasort/1G-input/part-m-00000 -files
> -blocks -locations
> Connecting to namenode via http://CHBM220:50070
> FSCK started by root (auth:SIMPLE) from /192.168.10.224 for path
> /alex/terasort/1G-input/part-m-00000 at Wed Dec 11 11:35:42 CST 2013
> /alex/terasort/1G-input/part-m-00000 100000000 bytes, 2 block(s):  OK
> 0. BP-50684181-192.168.10.220-1383638483950:blk_3717620888497075523_106232
> len=67108864 repl=3 [192.168.10.222:50010, 192.168.10.223:50010,
> 192.168.10.221:50010]
> 1. BP-50684181-192.168.10.220-1383638483950:blk_2504407693800874616_106252
> len=32891136 repl=3 [192.168.10.222:50010, 192.168.10.221:50010,
> 192.168.10.224:50010]

--
Harsh J