Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # dev - DataBlockScanner scan period


Copy link to this message
-
Re: DataBlockScanner scan period
Brian Bockelman 2010-10-14, 00:07
Hi Thanh,

The scan period is the period that hadoop *attempts* to complete an entire node scan.  That is, if it's set to 3 weeks, HDFS will try to scan each block once every 3 weeks.

Obviously, depending on the bandwidth you have made available to the scanning thread, you can specify impossibly small periods.

Brian

On Oct 13, 2010, at 7:01 PM, Thanh Do wrote:

> Hi again,
>
> Could any body explain to me about the scanning period
> policy of DataBlockScanner? That is who often it wake up
> and scan a block file.
> When looking at the code, I found
>
> static final long DEFAULT_SCAN_PERIOD_HOURS = 21*24L; // three weeks
>
>
> but definitely it does not wake up and pick a random block
> to verify every three weeks, right?
>
> Thanks a lot,
> Thanh