We are trying to secure our HDFS installation by blocking all the ports that HDFS requires to the outside world.
Unfortunately it's not possible to give our machines private IPs (... dont ask me why... ). So we were starting to
compile a list of ports that HDFS uses, so we can specifically block traffic to these ports. So far we found that we can
configure the following ports:

dfs.datanode.http.address � 50075
dfs.datanode.address � 50010
dfs.datanode.ipc.address � 50020

however we found via netstat -ltp that the HDFS datanode also listens on another random port and so far we've been
unable to determine what that port is used for and how to configure it to be on a fixed port. Can anyone help with this?

The read port of data node is random, but I think you can forbid the
port on namenode becuase each read operation should access namenode
first.

On Fri, Sep 3, 2010 at 12:53 AM, Stephan Gammeter
<[EMAIL PROTECTED]> wrote:
> We are trying to secure our HDFS installation by blocking all the ports that
> HDFS requires to the outside world. Unfortunately it's not possible to give
> our machines private IPs (... dont ask me why... ). So we were starting to
> compile a list of ports that HDFS uses, so we can specifically block traffic
> to these ports. So far we found that we can configure the following ports:
>
> dfs.datanode.http.address – 50075
> dfs.datanode.address – 50010
> dfs.datanode.ipc.address – 50020
>
> however we found via netstat -ltp that the HDFS datanode also listens on
> another random port and so far we've been unable to determine what that port
> is used for and how to configure it to be on a fixed port. Can anyone help
> with this?
>

--
Best Regards

Jeff Zhang

Hi Stephan,

Rather than specifically blocking these ports, why not use a default DENY
policy and explicitly allow the ones you'd like to the outside world (eg
ssh?) This seems a lot easier than tracking down the specific ports to deny.

Regarding the specific question, my guess is that it's the JMX remoting
port. Do you set -Dcom.sun.management.jmxremote as a java option in
hadoop-env.sh?

-Todd

On Fri, Sep 3, 2010 at 12:53 AM, Stephan Gammeter <
[EMAIL PROTECTED]> wrote:

> We are trying to secure our HDFS installation by blocking all the ports
> that HDFS requires to the outside world. Unfortunately it's not possible to
> give our machines private IPs (... dont ask me why... ). So we were starting
> to compile a list of ports that HDFS uses, so we can specifically block
> traffic to these ports. So far we found that we can configure the following
> ports:
>
> dfs.datanode.http.address – 50075
> dfs.datanode.address – 50010
> dfs.datanode.ipc.address – 50020
>
> however we found via netstat -ltp that the HDFS datanode also listens on
> another random port and so far we've been unable to determine what that port
> is used for and how to configure it to be on a fixed port. Can anyone help
> with this?
>

--
Todd Lipcon
Software Engineer, Cloudera