Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # dev >> Re: ReplicationTargetChooser has incorrect block placement comments


Copy link to this message
-
Re: ReplicationTargetChooser has incorrect block placement comments
Good to know! Please also consider improving the javadoc on that
method to reflect this, by logging a doc JIRA under HDFS :)

On Thu, Apr 4, 2013 at 12:33 PM, Mei Long <[EMAIL PROTECTED]> wrote:
> Harsh,
>
> Thank you! This is exactly what I was confused about. By looking at the code
> I was suspecting that it's not looking at rack locality for the second
> scenario (No DN on writer node.) But I was unable to confirm it without
> running the code.
>
> Your answer makes so much sense. Thank you!! :-) I'm much happier now.
>
> On Thu, Apr 4, 2013 at 2:14 AM, Harsh J <[EMAIL PROTECTED]> wrote:
>>
>> Hi Mei,
>>
>> These questions are best fit for the whole hdfs-dev@ group, which am
>> adding while responding back.
>>
>> You're correct that the local node being not chosen, a random node on
>> the same rack as the writer may be tried. However, if the writer node
>> itself had no local DN in the first place, a completely random node
>> across any rack may be selected.
>>
>> This is visible at
>>
>> http://svn.apache.org/viewvc/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java?view=markup
>> (~263).
>>
>> On Thu, Apr 4, 2013 at 12:53 AM, Mei Long <[EMAIL PROTECTED]> wrote:
>> > Hi Harsh,
>> >
>> > I found your post on JIRA regarding the comments. I find them very
>> > confusing
>> > as well. I'm trying to figure out the following comment:
>> >
>> >  * the 1st replica is placed on the local machine,
>> >  * otherwise a random datanode.
>> >
>> > Do you know when it says "otherwise a random datanode," does it mean any
>> > random datanode anywhere on the network or a random datanode on the same
>> > rack as the local machine? I've been looking at the code for an hour and
>> > I'm
>> > getting more confused with the comment and code in chooseLocalNode()
>> >
>> >   /* choose <i>localMachine</i> as the target.
>> >
>> >    * if <i>localMachine</i> is not availabe,
>> >
>> >    * choose a node on the same rack
>> >
>> >    * @return the choosen node
>> >
>> >    */
>> >
>> > Your help is much appreciated!
>> >
>> > Mei
>>
>>
>>
>> --
>> Harsh J
>
>

--
Harsh J