Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # dev >> What is the reason that DataNode should configured the http and streamer port with a number under 1024 in secure mode?


Copy link to this message
-
Re: 答复: What is the reason that DataNode should configured the http and streamer port with a number under 1024 in secure mode?
On Sun, Nov 18, 2012 at 8:28 PM, Xiaohan <[EMAIL PROTECTED]> wrote:

> Hi Aaron,
>
> Thanks for your kind reply. Your explanation is really helpful for me.
> Still left one question:
> >> However, in order for a user to get a malicious DN to register with the
> NN that user will need access to the DN keytab.
> Is there any difference between DN registration and normal NN RPC? I mean
> at the respect of authentication?
> As far as I know, there seem be no difference. So a client who can visit
> the NN (such as creating a file) also can make a registration to NN.
>

The DN->NN RPCs come in on a different port and different IPC interface
than the client RPCs. The DN->NN RPC only allows a single configured
principal (ie the DNs) to connect if security is enabled.

-Todd
> -----邮件原件-----
> 发件人: Aaron T. Myers [mailto:[EMAIL PROTECTED]]
> 发送时间: 2012年11月17日 3:41
> 收件人: [EMAIL PROTECTED]
> 主题: Re: What is the reason that DataNode should configured the http and
> streamer port with a number under 1024 in secure mode?
>
> Hi Xiaohan,
>
> You're right that if a malicious user can get a fake DN to register with
> the NN then the low ports don't matter. However, in order for a user to get
> a malicious DN to register with the NN that user will need access to the DN
> keytab. If the user has access to that, then all is lost anyway since by
> logging in with that keytab the user can act as an HDFS super user.
>
> The purpose of the requiring the low ports is to prevent the following
> scenario:
>
> 1) Malicious user finds a crashed DN process or somehow causes a DN process
> to crash.
> 2) Before the NN considers that DN dead (by default 10 minutes) the
> malicious user starts a fake DN process on the same (high) ports.
> 3) The NN continues to tell clients that it's OK to write to that DN that
> has just crashed for 10 minutes.
> 4) The malicious user steals all the data written to the crashed process in
> those 10 minutes.
>
> I hope this clears things up.
>
> --
> Aaron T. Myers
> Software Engineer, Cloudera
>
>
>
> On Fri, Nov 16, 2012 at 6:44 AM, Xiaohan <[EMAIL PROTECTED]> wrote:
>
> > Hi, guys.
> >
> > Now our cluster is moving to security mode. We find many difference with
> > the non-security, one is the starting of datanode. And I am not sure how
> it
> > works, so I send the email here to ask.
> > Secure mode must use jsvc liking tools to start datanode because it
> allows
> > the datanode listening the port under 1024 woring in not-root user.
> > I search the reason of using the port under 1024 with google, only findng
> > that Cloudera's CDH doc describes the reason, which is "DataNode must be
> > below 1024, because this provides part of the security mechanism to make
> it
> > impossible for a user to run a map task which impersonates a DataNode."
> > I try to configure the datanode's http port with 2004(the suggesting
> value
> > is 1004) which is above 1024, then starting it in secure mode. It result
> in
> > a failure of starting the one as expected. But I found the failure is
> > because of the DataNode itself check the number and throws the exception.
> > Since user to run map task may impersonate the DataNode, he could also
> > change the code of DataNode with avoiding the check in DataNode. When
> user
> > do it, it still impersonate the DataNode with a port above 1024, which a
> > non-root user could use and then application in map task could use.
> > Then I supposed that NN should also do the check, so I deleted the check
> > code in DataNode, configuring the http port with 2004, then starting
> > DataNode in secure mode. The DataNode starting successfully and the NN
> > accept the DataNode.
> > The data is also writed to the DataNode. Everything works well as the
> > DataNode is a normal one.
> >
> > Is it a defect? Or I 've missed something. If either of them, please let
> > me know. Thank you.
> >
>

--
Todd Lipcon
Software Engineer, Cloudera
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB