Dhaivat Pandya 2013-12-23, 04:11
Shekhar Sharma 2013-12-23, 04:28
Dhaivat Pandya 2013-12-23, 04:38
-Re: Datanode registration, port number
Dhaivat Pandya 2013-12-24, 02:24
Any other ideas?
On Sun, Dec 22, 2013 at 10:38 PM, Dhaivat Pandya <[EMAIL PROTECTED]>wrote:
> I understand that that is how the port is *later* retrieved, but how does
> the namenode know the port in the first place? i.e. if the datanode sends a
> packet to the namenode, how does the namenode know what port # the datanode
> server is running on?
> On Sun, Dec 22, 2013 at 10:28 PM, Shekhar Sharma <[EMAIL PROTECTED]>wrote:
>> Check the version file
>> During the startup Datanode sends out a heart beat signal and NN
>> assigns a storageID for it which is unique for every datanode.
>> Using the storage id it identify the ip and port of a datanode
>> Som Shekhar Sharma
>> On Mon, Dec 23, 2013 at 9:41 AM, Dhaivat Pandya <[EMAIL PROTECTED]>
>> > Hi,
>> > I'm currently trying to build a cache layer that should sit "on top" of
>> > datanode. Essentially, the namenode should know the port number of the
>> > cache layer instead of that of the datanode (since the namenode then
>> > this information to the default HDFS client). All of the communication
>> > between the datanode and the namenode currently flows through my cache
>> > layer (including heartbeats, etc.)
>> > *First question*: is there a way to tell the namenode where a datanode
>> > should be? Any way to trick it into thinking that the datanode is on a
>> > number where it actually isn't? As far as I can tell, the port number is
>> > obtained from the DatanodeId object; can this be set in the
>> > so that the port number derived is that of the cache layer?
>> > I spent quite a bit of time on the above question and I could not find
>> > sort of configuration option that would let me do that. So, I delved
>> > the HDFS source code and tracked down the DatanodeRegistration class.
>> > However, I can't seem to find out *how* the NameNode figures out the
>> > Datanode's port number or if I could somehow change the packets to
>> > the port number of cache layer? *Second question: *how does the namenode
>> > figure out a newly-registered Datanode's port number?
>> > Thank you,
>> > Dhaivat
Harsh J 2013-12-24, 05:46
Dhaivat Pandya 2013-12-24, 05:57
Andrew Wang 2013-12-30, 20:48
Dhaivat Pandya 2013-12-31, 20:35