Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> Re: issue about corrupt block test


Copy link to this message
-
Re: issue about corrupt block test
you are right ,but i only find meta file why no block data file?
# find /data -name 'blk_-5451264646515882190_106793*'
/data/dataspace/3/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/subdir39/blk_-5451264646515882190_106793.meta

# ls
/data/dataspace/3/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/subdir39/
blk_3810334848964580951              blk_4621466474283145207_106809.meta
blk_-5451264646515882190              blk_580162309124277323_106788.meta
blk_3810334848964580951_106801.meta  blk_516060569193828059
blk_-5451264646515882190_106793.meta
blk_4621466474283145207              blk_516060569193828059_106796.meta
blk_580162309124277323
On Wed, Dec 11, 2013 at 3:16 PM, Harsh J <[EMAIL PROTECTED]> wrote:

> Block files are not stored in a flat directory (to avoid FS limits of
> max files under a dir). Instead of looking for them right under
> finalized, issue a "find" query with the pattern instead and you
> should be able to spot it.
>
> On Wed, Dec 11, 2013 at 9:10 AM, ch huang <[EMAIL PROTECTED]> wrote:
> > hi,maillist:
> >              i try to corrupt a block of a file in my benchmark
> environment,
> > as the following command i find blk_2504407693800874616_106252 ,it's
> replica
> > on 192.168.10.224 is my target ,but i find all the datadir in
> 192.168.10.224
> > ,can not fine the datafile belongs to this replic ,why?
> >
> > # ls
> >
> /data/dataspace/1/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/blk_3717620888497075523_106232*
> > ls: cannot access
> >
> /data/dataspace/1/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/blk_3717620888497075523_106232*:
> > No such file or directory
> > [root@CHBM224 conf]# ls
> >
> /data/dataspace/1/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/blk_2504407693800874616_106252*
> > ls: cannot access
> >
> /data/dataspace/1/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/blk_2504407693800874616_106252*:
> > No such file or directory
> > [root@CHBM224 conf]# ls
> >
> /data/dataspace/2/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/blk_2504407693800874616_106252*
> > ls: cannot access
> >
> /data/dataspace/2/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/blk_2504407693800874616_106252*:
> > No such file or directory
> > [root@CHBM224 conf]# ls
> >
> /data/dataspace/3/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/blk_2504407693800874616_106252*
> > ls: cannot access
> >
> /data/dataspace/3/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/blk_2504407693800874616_106252*:
> > No such file or directory
> > [root@CHBM224 conf]# hdfs fsck /alex/terasort/1G-input/part-m-00000
> -files
> > -blocks -locations
> > Connecting to namenode via http://CHBM220:50070 <http://chbm220:50070/>
> > FSCK started by root (auth:SIMPLE) from /192.168.10.224 for path
> > /alex/terasort/1G-input/part-m-00000 at Wed Dec 11 11:35:42 CST 2013
> > /alex/terasort/1G-input/part-m-00000 100000000 bytes, 2 block(s):  OK
> > 0.
> BP-50684181-192.168.10.220-1383638483950:blk_3717620888497075523_106232
> > len=67108864 repl=3 [192.168.10.222:50010, 192.168.10.223:50010,
> > 192.168.10.221:50010]
> > 1.
> BP-50684181-192.168.10.220-1383638483950:blk_2504407693800874616_106252
> > len=32891136 repl=3 [192.168.10.222:50010, 192.168.10.221:50010,
> > 192.168.10.224:50010]
>
>
>
> --
> Harsh J
>
+
Harsh J 2013-12-11, 09:36