Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> Re: issue about corrupt block test


Copy link to this message
-
Re: issue about corrupt block test
you are right ,but i only find meta file why no block data file?
# find /data -name 'blk_-5451264646515882190_106793*'
/data/dataspace/3/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/subdir39/blk_-5451264646515882190_106793.meta

# ls
/data/dataspace/3/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/subdir39/
blk_3810334848964580951              blk_4621466474283145207_106809.meta
blk_-5451264646515882190              blk_580162309124277323_106788.meta
blk_3810334848964580951_106801.meta  blk_516060569193828059
blk_-5451264646515882190_106793.meta
blk_4621466474283145207              blk_516060569193828059_106796.meta
blk_580162309124277323
On Wed, Dec 11, 2013 at 3:16 PM, Harsh J <[EMAIL PROTECTED]> wrote:

> Block files are not stored in a flat directory (to avoid FS limits of
> max files under a dir). Instead of looking for them right under
> finalized, issue a "find" query with the pattern instead and you
> should be able to spot it.
>
> On Wed, Dec 11, 2013 at 9:10 AM, ch huang <[EMAIL PROTECTED]> wrote:
> > hi,maillist:
> >              i try to corrupt a block of a file in my benchmark
> environment,
> > as the following command i find blk_2504407693800874616_106252 ,it's
> replica
> > on 192.168.10.224 is my target ,but i find all the datadir in
> 192.168.10.224
> > ,can not fine the datafile belongs to this replic ,why?
> >
> > # ls
> >
> /data/dataspace/1/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/blk_3717620888497075523_106232*
> > ls: cannot access
> >
> /data/dataspace/1/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/blk_3717620888497075523_106232*:
> > No such file or directory
> > [root@CHBM224 conf]# ls
> >
> /data/dataspace/1/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/blk_2504407693800874616_106252*
> > ls: cannot access
> >
> /data/dataspace/1/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/blk_2504407693800874616_106252*:
> > No such file or directory
> > [root@CHBM224 conf]# ls
> >
> /data/dataspace/2/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/blk_2504407693800874616_106252*
> > ls: cannot access
> >
> /data/dataspace/2/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/blk_2504407693800874616_106252*:
> > No such file or directory
> > [root@CHBM224 conf]# ls
> >
> /data/dataspace/3/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/blk_2504407693800874616_106252*
> > ls: cannot access
> >
> /data/dataspace/3/current/BP-50684181-192.168.10.220-1383638483950/current/finalized/blk_2504407693800874616_106252*:
> > No such file or directory
> > [root@CHBM224 conf]# hdfs fsck /alex/terasort/1G-input/part-m-00000
> -files
> > -blocks -locations
> > Connecting to namenode via http://CHBM220:50070 <http://chbm220:50070/>
> > FSCK started by root (auth:SIMPLE) from /192.168.10.224 for path
> > /alex/terasort/1G-input/part-m-00000 at Wed Dec 11 11:35:42 CST 2013
> > /alex/terasort/1G-input/part-m-00000 100000000 bytes, 2 block(s):  OK
> > 0.
> BP-50684181-192.168.10.220-1383638483950:blk_3717620888497075523_106232
> > len=67108864 repl=3 [192.168.10.222:50010, 192.168.10.223:50010,
> > 192.168.10.221:50010]
> > 1.
> BP-50684181-192.168.10.220-1383638483950:blk_2504407693800874616_106252
> > len=32891136 repl=3 [192.168.10.222:50010, 192.168.10.221:50010,
> > 192.168.10.224:50010]
>
>
>
> --
> Harsh J
>
+
Harsh J 2013-12-11, 09:36
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB