Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # dev - Re: Review Request: DFSClient.getBlockLocations returns BlockLocations with no indication that the corresponding blocks are corrupt


Copy link to this message
-
Re: Review Request: DFSClient.getBlockLocations returns BlockLocations with no indication that the corresponding blocks are corrupt
Arun C Murthy 2010-11-03, 20:27
Can we get RB to send these to jira? Having comments here and on jira  
is very confusing...

On Nov 3, 2010, at 11:11 AM, Ramkumar Vadali wrote:

>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27/#review27
> -----------------------------------------------------------
>
>
>
> http://svn.apache.org/repos/asf/hadoop/hdfs/trunk/src/java/org/apache/hadoop/hdfs/DFSUtil.java
> <https://reviews.apache.org/r/27/#comment17>
>
>    It should be sufficient to pass blk.isCorrupt() here. The client  
> can check the number of locations based on the remaining information.
>
>
> - Ramkumar
>
>
> On 2010-11-02 21:12:28, Patrick Kling wrote:
>>
>> -----------------------------------------------------------
>> This is an automatically generated e-mail. To reply, visit:
>> https://reviews.apache.org/r/27/
>> -----------------------------------------------------------
>>
>> (Updated 2010-11-02 21:12:28)
>>
>>
>> Review request for hadoop-hdfs.
>>
>>
>> Summary
>> -------
>>
>> DFSClient.getBlockLocations returns BlockLocations with no  
>> indication that the corresponding blocks are corrupt
>>
>> When there are no uncorrupted replicas of a block,  
>> FSNamesystem.getBlockLocations returns LocatedBlocks corresponding  
>> to corrupt blocks. When DFSClient converts these to BlockLocations,  
>> the information that the corresponding block is corrupt is lost. We  
>> should add a field to BlockLocation to indicate whether the  
>> corresponding block is corrupt in order to warn the client that  
>> reading this block will fail. This would be especially useful for  
>> tools such as RAID FSCK, which could then easily inspect whether  
>> data or parity blocks are corrupted without having to make direct  
>> RPC calls
>>
>>
>> This addresses bug HDFS-1483.
>>    https://issues.apache.org/jira/browse/HDFS-1483
>>
>>
>> Diffs
>> -----
>>
>>  http://svn.apache.org/repos/asf/hadoop/hdfs/trunk/src/java/org/apache/hadoop/hdfs/DFSUtil.java
>>  1028386
>>  http://svn.apache.org/repos/asf/hadoop/hdfs/trunk/src/test/hdfs/org/apache/hadoop/hdfs/TestDFSUtil.java
>>  PRE-CREATION
>>
>> Diff: https://reviews.apache.org/r/27/diff
>>
>>
>> Testing
>> -------
>>
>> TestDFSUtil
>>
>>
>> Thanks,
>>
>> Patrick
>>
>>
>