Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # dev - Datanode registration, port number

Copy link to this message
Re: Datanode registration, port number
Dhaivat Pandya 2013-12-23, 04:38
I understand that that is how the port is *later* retrieved, but how does
the namenode know the port in the first place? i.e. if the datanode sends a
packet to the namenode, how does the namenode know what port # the datanode
server is running on?
On Sun, Dec 22, 2013 at 10:28 PM, Shekhar Sharma <[EMAIL PROTECTED]>wrote:

> Check the version file
> During the startup Datanode sends out a heart beat signal and NN
> assigns a storageID for it which is unique for every datanode.
> Using the storage id it identify the ip and port of a datanode
> Regards,
> Som Shekhar Sharma
> +91-8197243810
> On Mon, Dec 23, 2013 at 9:41 AM, Dhaivat Pandya <[EMAIL PROTECTED]>
> wrote:
> > Hi,
> >
> > I'm currently trying to build a cache layer that should sit "on top" of
> the
> > datanode. Essentially, the namenode should know the port number of the
> > cache layer instead of that of the datanode (since the namenode then
> relays
> > this information to the default HDFS client). All of the communication
> > between the datanode and the namenode currently flows through my cache
> > layer (including heartbeats, etc.)
> >
> > *First question*: is there a way to tell the namenode where a datanode
> > should be? Any way to trick it into thinking that the datanode is on a
> port
> > number where it actually isn't? As far as I can tell, the port number is
> > obtained from the DatanodeId object; can this be set in the configuration
> > so that the port number derived is that of the cache layer?
> >
> > I spent quite a bit of time on the above question and I could not find
> any
> > sort of configuration option that would let me do that. So, I delved into
> > the HDFS source code and tracked down the DatanodeRegistration class.
> > However, I can't seem to find out *how* the NameNode figures out the
> > Datanode's port number or if I could somehow change the packets to
> reflect
> > the port number of cache layer? *Second question: *how does the namenode
> > figure out a newly-registered Datanode's port number?
> >
> > Thank you,
> >
> > Dhaivat