Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> HDFS problem in hadoop 0.20.203


Copy link to this message
-
Re: HDFS problem in hadoop 0.20.203
Hi,
 Did you formatted the HDFS?

On Tue, Feb 21, 2012 at 7:40 PM, Shi Yu <[EMAIL PROTECTED]> wrote:

> Hi Hadoopers,
>
> We are experiencing a strange problem on Hadoop 0.20.203
>
> Our cluster has 58 nodes, everything is started from a fresh
> HDFS (we deleted all local folders on datanodes and
> reformatted the namenode).  After running some small jobs, the
> HDFS becomes behaving abnormally and the jobs become very
> slow.  The namenode log is crushed by Gigabytes of errors like
> is:
>
> 2012-02-21 00:00:38,632 INFO
> org.apache.hadoop.hdfs.StateChange: BLOCK*
> NameSystem.addToInvalidates: blk_4524177823306792294 is added
> to invalidSet of 10.105.19.31:50010
> 2012-02-21 00:00:38,632 INFO
> org.apache.hadoop.hdfs.StateChange: BLOCK*
> NameSystem.addToInvalidates: blk_4524177823306792294 is added
> to invalidSet of 10.105.19.18:50010
> 2012-02-21 00:00:38,632 INFO
> org.apache.hadoop.hdfs.StateChange: BLOCK*
> NameSystem.addToInvalidates: blk_4524177823306792294 is added
> to invalidSet of 10.105.19.32:50010
> 2012-02-21 00:00:38,632 INFO
> org.apache.hadoop.hdfs.StateChange: BLOCK*
> NameSystem.addToInvalidates: blk_2884522252507300332 is added
> to invalidSet of 10.105.19.35:50010
> 2012-02-21 00:00:38,632 INFO
> org.apache.hadoop.hdfs.StateChange: BLOCK*
> NameSystem.addToInvalidates: blk_2884522252507300332 is added
> to invalidSet of 10.105.19.27:50010
> 2012-02-21 00:00:38,632 INFO
> org.apache.hadoop.hdfs.StateChange: BLOCK*
> NameSystem.addToInvalidates: blk_2884522252507300332 is added
> to invalidSet of 10.105.19.33:50010
> 2012-02-21 00:00:38,632 INFO
> org.apache.hadoop.hdfs.StateChange: BLOCK*
> NameSystem.addStoredBlock: blockMap updated:
> 10.105.19.21:50010 is added to blk_-
> 6843171124277753504_2279882 size 124490
> 2012-02-21 00:00:38,632 INFO
> org.apache.hadoop.hdfs.StateChange: BLOCK*
> NameSystem.allocateBlock:
> /syu/output/naive/iter5_partout1/_temporary/_attempt_201202202
> 043_0013_m_000313_0/result_stem-m-00313. blk_-
> 6379064588594672168_2279890
> 2012-02-21 00:00:38,633 INFO
> org.apache.hadoop.hdfs.StateChange: BLOCK*
> NameSystem.addStoredBlock: blockMap updated:
> 10.105.19.26:50010 is added to blk_5338983375361999760_2279887
> size 1476
> 2012-02-21 00:00:38,633 INFO
> org.apache.hadoop.hdfs.StateChange: BLOCK*
> NameSystem.addStoredBlock: blockMap updated:
> 10.105.19.29:50010 is added to blk_-977828927900581074_2279887
> size 13818
> 2012-02-21 00:00:38,633 INFO
> org.apache.hadoop.hdfs.StateChange: DIR*
> NameSystem.completeFile: file
> /syu/output/naive/iter5_partout1/_temporary/_attempt_201202202
> 043_0013_m_000364_0/result_stem-m-00364 is closed by
> DFSClient_attempt_201202202043_0013_m_000364_0
> 2012-02-21 00:00:38,633 INFO
> org.apache.hadoop.hdfs.StateChange: BLOCK*
> NameSystem.addStoredBlock: blockMap updated:
> 10.105.19.23:50010 is added to blk_5338983375361999760_2279887
> size 1476
> 2012-02-21 00:00:38,633 INFO
> org.apache.hadoop.hdfs.StateChange: BLOCK*
> NameSystem.addStoredBlock: blockMap updated:
> 10.105.19.20:50010 is added to blk_5338983375361999760_2279887
> size 1476
> 2012-02-21 00:00:38,633 INFO
> org.apache.hadoop.hdfs.StateChange: BLOCK*
> NameSystem.allocateBlock:
> /syu/output/naive/iter5_partout1/_temporary/_attempt_201202202
> 043_0013_m_000364_0/result_suffix-m-00364.
> blk_1921685366929756336_2279890
> 2012-02-21 00:00:38,634 INFO
> org.apache.hadoop.hdfs.StateChange: DIR*
> NameSystem.completeFile: file
> /syu/output/naive/iter5_partout1/_temporary/_attempt_201202202
> 043_0013_m_000279_0/result_suffix-m-00279 is closed by
> DFSClient_attempt_201202202043_0013_m_000279_0
> 2012-02-21 00:00:38,635 INFO
> org.apache.hadoop.hdfs.StateChange: BLOCK*
> NameSystem.addToInvalidates: blk_495061820035691700 is added
> to invalidSet of 10.105.19.20:50010
> 2012-02-21 00:00:38,635 INFO
> org.apache.hadoop.hdfs.StateChange: BLOCK*
> NameSystem.addToInvalidates: blk_495061820035691700 is added
> to invalidSet of 10.105.19.25:50010
> 2012-02-21 00:00:38,635 INFO
Join me at http://hadoopworkshop.eventbrite.com/