I recently got a mini cluster corrupted after my inappropriate process.
This mini cluster's dfs.replication was set to 1. After irregular restart of OS, I cannot wait to leave safemode, the block ratio is 0.9862, < 0.999. In the http://ip:50075/blockScannerReport, I notice there is rate limit to 1MB. It will verify the blocks for long time.
So I "hadoop dfdsadmin safemode leave", and then I got blocks missing.
My question is: Why should we limit the rate in DataBlockScanner while the cluster is still starting up or still in safemode?
I read the source code of DataBlockScanner.java, there is no parameter to change the rate limit. It seams to be 1MB to 8MB always. -- Davey Yan