Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - Re: issue about corrupt block test


Copy link to this message
-
Re: issue about corrupt block test
Harsh J 2013-12-11, 09:36
Your find expression includes the genstamp, try it without the genstamp:

find /data -name 'blk_-5451264646515882190*'

On Wed, Dec 11, 2013 at 2:46 PM, ch huang <[EMAIL PROTECTED]> wrote:
>
> you are right ,but i only find meta file why no block data file?
> # find /data -name 'blk_-5451264646515882190_106793*'
> /data/dataspace/3/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/subdir39/blk_-5451264646515882190_106793.meta
>
> # ls
> /data/dataspace/3/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/subdir39/
> blk_3810334848964580951              blk_4621466474283145207_106809.meta
> blk_-5451264646515882190              blk_580162309124277323_106788.meta
> blk_3810334848964580951_106801.meta  blk_516060569193828059
> blk_-5451264646515882190_106793.meta
> blk_4621466474283145207              blk_516060569193828059_106796.meta
> blk_580162309124277323
>
>
> On Wed, Dec 11, 2013 at 3:16 PM, Harsh J <[EMAIL PROTECTED]> wrote:
>>
>> Block files are not stored in a flat directory (to avoid FS limits of
>> max files under a dir). Instead of looking for them right under
>> finalized, issue a "find" query with the pattern instead and you
>> should be able to spot it.
>>
>> On Wed, Dec 11, 2013 at 9:10 AM, ch huang <[EMAIL PROTECTED]> wrote:
>> > hi,maillist:
>> >              i try to corrupt a block of a file in my benchmark
>> > environment,
>> > as the following command i find blk_2504407693800874616_106252 ,it's
>> > replica
>> > on 192.168.10.224 is my target ,but i find all the datadir in
>> > 192.168.10.224
>> > ,can not fine the datafile belongs to this replic ,why?
>> >
>> > # ls
>> >
>> > /data/dataspace/1/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/blk_3717620888497075523_106232*
>> > ls: cannot access
>> >
>> > /data/dataspace/1/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/blk_3717620888497075523_106232*:
>> > No such file or directory
>> > [root@CHBM224 conf]# ls
>> >
>> > /data/dataspace/1/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/blk_2504407693800874616_106252*
>> > ls: cannot access
>> >
>> > /data/dataspace/1/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/blk_2504407693800874616_106252*:
>> > No such file or directory
>> > [root@CHBM224 conf]# ls
>> >
>> > /data/dataspace/2/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/blk_2504407693800874616_106252*
>> > ls: cannot access
>> >
>> > /data/dataspace/2/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/blk_2504407693800874616_106252*:
>> > No such file or directory
>> > [root@CHBM224 conf]# ls
>> >
>> > /data/dataspace/3/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/blk_2504407693800874616_106252*
>> > ls: cannot access
>> >
>> > /data/dataspace/3/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/blk_2504407693800874616_106252*:
>> > No such file or directory
>> > [root@CHBM224 conf]# hdfs fsck /alex/terasort/1G-input/part-m-00000
>> > -files
>> > -blocks -locations
>> > Connecting to namenode via http://CHBM220:50070
>> > FSCK started by root (auth:SIMPLE) from /192.168.10.224 for path
>> > /alex/terasort/1G-input/part-m-00000 at Wed Dec 11 11:35:42 CST 2013
>> > /alex/terasort/1G-input/part-m-00000 100000000 bytes, 2 block(s):  OK
>> > 0.
>> > BP-50684181-192.168.10.220-1383638483950:blk_3717620888497075523_106232
>> > len=67108864 repl=3 [192.168.10.222:50010, 192.168.10.223:50010,
>> > 192.168.10.221:50010]
>> > 1.
>> > BP-50684181-192.168.10.220-1383638483950:blk_2504407693800874616_106252
>> > len=32891136 repl=3 [192.168.10.222:50010, 192.168.10.221:50010,
>> > 192.168.10.224:50010]
>>
>>
>>
>> --
>> Harsh J
>
>

--
Harsh J