Hi all,

   I configured like below in hdfs-site.xml:

<property>
  <name>dfs.namenode.kerberos.principal</name>
  <value>nn/_HOST@site</value>
</property>
<property>
    <name>dfs.web.authentication.kerberos.principal</name>
    <value>nn/_HOST@site</value>
</property>
   When  start up namenode, I found, namenode will use principal : nn/167-52-0-56@site to login, but the http server will use nn/167-52-0-56.site@site<mailto:nn/167-52-0-56.site@site> to lgin,  so it start failed.

I checked the code,

Namenode will use socAddr.getHostName() to get hostname in org.apache.hadoop.hdfs.server.namenode.NameNode.loginAsNameNodeUser.
But httpserver 's default hostname is 0.0.0.0, so in org.apache.hadoop.security.SecurityUtil.replacePattern, it will get the hostname by invoking getLocalHostName,there it use getCanonicalHostName(),

I think this inconsistent is wrong,  can someone confirm this? Need raise one bug ?

Thanks

what version of hadoop are you using?

also

dfs.web.authentication.kerberos.principal should be set to HTTP/[EMAIL PROTECTED]

--
Arpit Gupta
Hortonworks Inc.
http://hortonworks.com/

On Jul 26, 2012, at 6:11 PM, Wangwenli <[EMAIL PROTECTED]> wrote:

> Hi all,
>
>   I configured like below in hdfs-site.xml:
>
> <property>
>  <name>dfs.namenode.kerberos.principal</name>
>  <value>nn/_HOST@site</value>
> </property>
>
>
> <property>
>    <name>dfs.web.authentication.kerberos.principal</name>
>    <value>nn/_HOST@site</value>
> </property>
>
>
>   When  start up namenode, I found, namenode will use principal : nn/167-52-0-56@site to login, but the http server will use nn/167-52-0-56.site@site<mailto:nn/167-52-0-56.site@site> to lgin,  so it start failed.
>
> I checked the code,
>
> Namenode will use socAddr.getHostName() to get hostname in org.apache.hadoop.hdfs.server.namenode.NameNode.loginAsNameNodeUser.
>
>
> But httpserver 's default hostname is 0.0.0.0, so in org.apache.hadoop.security.SecurityUtil.replacePattern, it will get the hostname by invoking getLocalHostName,there it use getCanonicalHostName(),
>
> I think this inconsistent is wrong,  can someone confirm this? Need raise one bug ?
>
> Thanks
>

Thank yours response.
I am using hadoop-2.0.0-alpha from apache site.  In which version it should configure with HTTP/[EMAIL PROTECTED]?  I think not in hadoop-2.0.0-alpha. Because I login successful with other principal, pls refer below log:

2012-07-23 22:48:17,303 INFO org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler: Login using keytab /home/hdfs/keytab/nn.service.keytab, for principal nn/167-52-0-56.site@site
2012-07-23 22:48:17,310 INFO org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler: Initialized, principal [nn/167-52-0-56.site@site] from keytab [/home/hdfs/keytab/nn.service.keytab]
-----邮件原件-----
发件人: Arpit Gupta [mailto:[EMAIL PROTECTED]]
发送时间: 2012年7月27日 9:22
收件人: [EMAIL PROTECTED]
主题: Re: regarding _HOST token replacement in security hadoop

what version of hadoop are you using?

also

dfs.web.authentication.kerberos.principal should be set to HTTP/[EMAIL PROTECTED]

--
Arpit Gupta
Hortonworks Inc.
http://hortonworks.com/

On Jul 26, 2012, at 6:11 PM, Wangwenli <[EMAIL PROTECTED]> wrote:

> Hi all,
>
>   I configured like below in hdfs-site.xml:
>
> <property>
>  <name>dfs.namenode.kerberos.principal</name>
>  <value>nn/_HOST@site</value>
> </property>
>
>
> <property>
>    <name>dfs.web.authentication.kerberos.principal</name>
>    <value>nn/_HOST@site</value>
> </property>
>
>
>   When  start up namenode, I found, namenode will use principal : nn/167-52-0-56@site to login, but the http server will use nn/167-52-0-56.site@site<mailto:nn/167-52-0-56.site@site> to lgin,  so it start failed.
>
> I checked the code,
>
> Namenode will use socAddr.getHostName() to get hostname in org.apache.hadoop.hdfs.server.namenode.NameNode.loginAsNameNodeUser.
>
>
> But httpserver 's default hostname is 0.0.0.0, so in org.apache.hadoop.security.SecurityUtil.replacePattern, it will get the hostname by invoking getLocalHostName,there it use getCanonicalHostName(),
>
> I think this inconsistent is wrong,  can someone confirm this? Need raise one bug ?
>
> Thanks
>

you need to use HTTP/[EMAIL PROTECTED] as that is the principal needed by spnego. So you would need create the HTTP/_HOST principal and add it to the same keytab (/home/hdfs/keytab/nn.service.keytab).

--
Arpit Gupta
Hortonworks Inc.
http://hortonworks.com/

On Jul 26, 2012, at 6:54 PM, Wangwenli <[EMAIL PROTECTED]> wrote:

> Thank yours response.
> I am using hadoop-2.0.0-alpha from apache site.  In which version it should configure with HTTP/[EMAIL PROTECTED]?  I think not in hadoop-2.0.0-alpha. Because I login successful with other principal, pls refer below log:
>
> 2012-07-23 22:48:17,303 INFO org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler: Login using keytab /home/hdfs/keytab/nn.service.keytab, for principal nn/167-52-0-56.site@site
> 2012-07-23 22:48:17,310 INFO org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler: Initialized, principal [nn/167-52-0-56.site@site] from keytab [/home/hdfs/keytab/nn.service.keytab]
>
>
> -----邮件原件-----
> 发件人: Arpit Gupta [mailto:[EMAIL PROTECTED]]
> 发送时间: 2012年7月27日 9:22
> 收件人: [EMAIL PROTECTED]
> 主题: Re: regarding _HOST token replacement in security hadoop
>
> what version of hadoop are you using?
>
> also
>
> dfs.web.authentication.kerberos.principal should be set to HTTP/[EMAIL PROTECTED]
>
> --
> Arpit Gupta
> Hortonworks Inc.
> http://hortonworks.com/
>
> On Jul 26, 2012, at 6:11 PM, Wangwenli <[EMAIL PROTECTED]> wrote:
>
>> Hi all,
>>
>>  I configured like below in hdfs-site.xml:
>>
>> <property>
>> <name>dfs.namenode.kerberos.principal</name>
>> <value>nn/_HOST@site</value>
>> </property>
>>
>>
>> <property>
>>   <name>dfs.web.authentication.kerberos.principal</name>
>>   <value>nn/_HOST@site</value>
>> </property>
>>
>>
>>  When  start up namenode, I found, namenode will use principal : nn/167-52-0-56@site to login, but the http server will use nn/167-52-0-56.site@site<mailto:nn/167-52-0-56.site@site> to lgin,  so it start failed.
>>
>> I checked the code,
>>
>> Namenode will use socAddr.getHostName() to get hostname in org.apache.hadoop.hdfs.server.namenode.NameNode.loginAsNameNodeUser.
>>
>>
>> But httpserver 's default hostname is 0.0.0.0, so in org.apache.hadoop.security.SecurityUtil.replacePattern, it will get the hostname by invoking getLocalHostName,there it use getCanonicalHostName(),
>>
>> I think this inconsistent is wrong,  can someone confirm this? Need raise one bug ?
>>
>> Thanks
>>
>

Could you spent one minute to check whether below code will cause issue or not?

In org.apache.hadoop.hdfs.server.namenode.NameNode.loginAsNameNodeUser(), it use socAddr.getHostName() to get _HOST,
But in org.apache.hadoop.security.SecurityUtil.replacePattern(), in getLocalHostName(), it use getCanonicalHostName() to get _HOST

Meanwhile I will check what you said. Thank you~
-----邮件原件-----
发件人: Arpit Gupta [mailto:[EMAIL PROTECTED]]
发送时间: 2012年7月27日 10:03
收件人: [EMAIL PROTECTED]
主题: Re: regarding _HOST token replacement in security hadoop

you need to use HTTP/[EMAIL PROTECTED] as that is the principal needed by spnego. So you would need create the HTTP/_HOST principal and add it to the same keytab (/home/hdfs/keytab/nn.service.keytab).

--
Arpit Gupta
Hortonworks Inc.
http://hortonworks.com/

On Jul 26, 2012, at 6:54 PM, Wangwenli <[EMAIL PROTECTED]> wrote:

> Thank yours response.
> I am using hadoop-2.0.0-alpha from apache site.  In which version it should configure with HTTP/[EMAIL PROTECTED]?  I think not in hadoop-2.0.0-alpha. Because I login successful with other principal, pls refer below log:
>
> 2012-07-23 22:48:17,303 INFO org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler: Login using keytab /home/hdfs/keytab/nn.service.keytab, for principal nn/167-52-0-56.site@site
> 2012-07-23 22:48:17,310 INFO org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler: Initialized, principal [nn/167-52-0-56.site@site] from keytab [/home/hdfs/keytab/nn.service.keytab]
>
>
> -----邮件原件-----
> 发件人: Arpit Gupta [mailto:[EMAIL PROTECTED]]
> 发送时间: 2012年7月27日 9:22
> 收件人: [EMAIL PROTECTED]
> 主题: Re: regarding _HOST token replacement in security hadoop
>
> what version of hadoop are you using?
>
> also
>
> dfs.web.authentication.kerberos.principal should be set to HTTP/[EMAIL PROTECTED]
>
> --
> Arpit Gupta
> Hortonworks Inc.
> http://hortonworks.com/
>
> On Jul 26, 2012, at 6:11 PM, Wangwenli <[EMAIL PROTECTED]> wrote:
>
>> Hi all,
>>
>>  I configured like below in hdfs-site.xml:
>>
>> <property>
>> <name>dfs.namenode.kerberos.principal</name>
>> <value>nn/_HOST@site</value>
>> </property>
>>
>>
>> <property>
>>   <name>dfs.web.authentication.kerberos.principal</name>
>>   <value>nn/_HOST@site</value>
>> </property>
>>
>>
>>  When  start up namenode, I found, namenode will use principal : nn/167-52-0-56@site to login, but the http server will use nn/167-52-0-56.site@site<mailto:nn/167-52-0-56.site@site> to lgin,  so it start failed.
>>
>> I checked the code,
>>
>> Namenode will use socAddr.getHostName() to get hostname in org.apache.hadoop.hdfs.server.namenode.NameNode.loginAsNameNodeUser.
>>
>>
>> But httpserver 's default hostname is 0.0.0.0, so in org.apache.hadoop.security.SecurityUtil.replacePattern, it will get the hostname by invoking getLocalHostName,there it use getCanonicalHostName(),
>>
>> I think this inconsistent is wrong,  can someone confirm this? Need raise one bug ?
>>
>> Thanks
>>
>

That does seem to be valid issue. Could you log a jira for it.

Thanks
On Thu, Jul 26, 2012 at 7:32 PM, Wangwenli <[EMAIL PROTECTED]> wrote:

> Could you spent one minute to check whether below code will cause issue or
> not?
>
> In org.apache.hadoop.hdfs.server.namenode.NameNode.loginAsNameNodeUser(),
> it use socAddr.getHostName() to get _HOST,
> But in org.apache.hadoop.security.SecurityUtil.replacePattern(), in
> getLocalHostName(), it use getCanonicalHostName() to get _HOST
>
> Meanwhile I will check what you said. Thank you~
>
>
> -----邮件原件-----
> 发件人: Arpit Gupta [mailto:[EMAIL PROTECTED]]
> 发送时间: 2012年7月27日 10:03
> 收件人: [EMAIL PROTECTED]
> 主题: Re: regarding _HOST token replacement in security hadoop
>
> you need to use HTTP/[EMAIL PROTECTED] as that is the principal needed by
> spnego. So you would need create the HTTP/_HOST principal and add it to the
> same keytab (/home/hdfs/keytab/nn.service.keytab).
>
> --
> Arpit Gupta
> Hortonworks Inc.
> http://hortonworks.com/
>
> On Jul 26, 2012, at 6:54 PM, Wangwenli <[EMAIL PROTECTED]> wrote:
>
> > Thank yours response.
> > I am using hadoop-2.0.0-alpha from apache site.  In which version it
> should configure with HTTP/[EMAIL PROTECTED]?  I think not in
> hadoop-2.0.0-alpha. Because I login successful with other principal, pls
> refer below log:
> >
> > 2012-07-23 22:48:17,303 INFO
> org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler:
> Login using keytab /home/hdfs/keytab/nn.service.keytab, for principal
> nn/167-52-0-56.site@site
> > 2012-07-23 22:48:17,310 INFO
> org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler:
> Initialized, principal [nn/167-52-0-56.site@site] from keytab
> [/home/hdfs/keytab/nn.service.keytab]
> >
> >
> > -----邮件原件-----
> > 发件人: Arpit Gupta [mailto:[EMAIL PROTECTED]]
> > 发送时间: 2012年7月27日 9:22
> > 收件人: [EMAIL PROTECTED]
> > 主题: Re: regarding _HOST token replacement in security hadoop
> >
> > what version of hadoop are you using?
> >
> > also
> >
> > dfs.web.authentication.kerberos.principal should be set to HTTP/_
> [EMAIL PROTECTED]
> >
> > --
> > Arpit Gupta
> > Hortonworks Inc.
> > http://hortonworks.com/
> >
> > On Jul 26, 2012, at 6:11 PM, Wangwenli <[EMAIL PROTECTED]> wrote:
> >
> >> Hi all,
> >>
> >>  I configured like below in hdfs-site.xml:
> >>
> >> <property>
> >> <name>dfs.namenode.kerberos.principal</name>
> >> <value>nn/_HOST@site</value>
> >> </property>
> >>
> >>
> >> <property>
> >>   <name>dfs.web.authentication.kerberos.principal</name>
> >>   <value>nn/_HOST@site</value>
> >> </property>
> >>
> >>
> >>  When  start up namenode, I found, namenode will use principal :
> nn/167-52-0-56@site to login, but the http server will use
> nn/167-52-0-56.site@site<mailto:nn/167-52-0-56.site@site> to lgin,  so it
> start failed.
> >>
> >> I checked the code,
> >>
> >> Namenode will use socAddr.getHostName() to get hostname in
> org.apache.hadoop.hdfs.server.namenode.NameNode.loginAsNameNodeUser.
> >>
> >>
> >> But httpserver 's default hostname is 0.0.0.0, so in
> org.apache.hadoop.security.SecurityUtil.replacePattern, it will get the
> hostname by invoking getLocalHostName,there it use getCanonicalHostName(),
> >>
> >> I think this inconsistent is wrong,  can someone confirm this? Need
> raise one bug ?
> >>
> >> Thanks
> >>
> >
>
>

What do you have set as the fs.defaultFS in your configuration? Make sure
that that is a fully-qualified domain name.

--
Aaron T. Myers
Software Engineer, Cloudera

On Fri, Jul 27, 2012 at 1:57 PM, Arpit Gupta <[EMAIL PROTECTED]> wrote:

> That does seem to be valid issue. Could you log a jira for it.
>
> Thanks
>
>
> On Thu, Jul 26, 2012 at 7:32 PM, Wangwenli <[EMAIL PROTECTED]> wrote:
>
> > Could you spent one minute to check whether below code will cause issue
> or
> > not?
> >
> > In org.apache.hadoop.hdfs.server.namenode.NameNode.loginAsNameNodeUser(),
> > it use socAddr.getHostName() to get _HOST,
> > But in org.apache.hadoop.security.SecurityUtil.replacePattern(), in
> > getLocalHostName(), it use getCanonicalHostName() to get _HOST
> >
> > Meanwhile I will check what you said. Thank you~
> >
> >
> > -----邮件原件-----
> > 发件人: Arpit Gupta [mailto:[EMAIL PROTECTED]]
> > 发送时间: 2012年7月27日 10:03
> > 收件人: [EMAIL PROTECTED]
> > 主题: Re: regarding _HOST token replacement in security hadoop
> >
> > you need to use HTTP/[EMAIL PROTECTED] as that is the principal needed by
> > spnego. So you would need create the HTTP/_HOST principal and add it to
> the
> > same keytab (/home/hdfs/keytab/nn.service.keytab).
> >
> > --
> > Arpit Gupta
> > Hortonworks Inc.
> > http://hortonworks.com/
> >
> > On Jul 26, 2012, at 6:54 PM, Wangwenli <[EMAIL PROTECTED]> wrote:
> >
> > > Thank yours response.
> > > I am using hadoop-2.0.0-alpha from apache site.  In which version it
> > should configure with HTTP/[EMAIL PROTECTED]?  I think not in
> > hadoop-2.0.0-alpha. Because I login successful with other principal, pls
> > refer below log:
> > >
> > > 2012-07-23 22:48:17,303 INFO
> >
> org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler:
> > Login using keytab /home/hdfs/keytab/nn.service.keytab, for principal
> > nn/167-52-0-56.site@site
> > > 2012-07-23 22:48:17,310 INFO
> >
> org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler:
> > Initialized, principal [nn/167-52-0-56.site@site] from keytab
> > [/home/hdfs/keytab/nn.service.keytab]
> > >
> > >
> > > -----邮件原件-----
> > > 发件人: Arpit Gupta [mailto:[EMAIL PROTECTED]]
> > > 发送时间: 2012年7月27日 9:22
> > > 收件人: [EMAIL PROTECTED]
> > > 主题: Re: regarding _HOST token replacement in security hadoop
> > >
> > > what version of hadoop are you using?
> > >
> > > also
> > >
> > > dfs.web.authentication.kerberos.principal should be set to HTTP/_
> > [EMAIL PROTECTED]
> > >
> > > --
> > > Arpit Gupta
> > > Hortonworks Inc.
> > > http://hortonworks.com/
> > >
> > > On Jul 26, 2012, at 6:11 PM, Wangwenli <[EMAIL PROTECTED]> wrote:
> > >
> > >> Hi all,
> > >>
> > >>  I configured like below in hdfs-site.xml:
> > >>
> > >> <property>
> > >> <name>dfs.namenode.kerberos.principal</name>
> > >> <value>nn/_HOST@site</value>
> > >> </property>
> > >>
> > >>
> > >> <property>
> > >>   <name>dfs.web.authentication.kerberos.principal</name>
> > >>   <value>nn/_HOST@site</value>
> > >> </property>
> > >>
> > >>
> > >>  When  start up namenode, I found, namenode will use principal :
> > nn/167-52-0-56@site to login, but the http server will use
> > nn/167-52-0-56.site@site<mailto:nn/167-52-0-56.site@site> to lgin,  so
> it
> > start failed.
> > >>
> > >> I checked the code,
> > >>
> > >> Namenode will use socAddr.getHostName() to get hostname in
> > org.apache.hadoop.hdfs.server.namenode.NameNode.loginAsNameNodeUser.
> > >>
> > >>
> > >> But httpserver 's default hostname is 0.0.0.0, so in
> > org.apache.hadoop.security.SecurityUtil.replacePattern, it will get the
> > hostname by invoking getLocalHostName,there it use
> getCanonicalHostName(),
> > >>
> > >> I think this inconsistent is wrong,  can someone confirm this? Need
> > raise one bug ?
> > >>
> > >> Thanks
> > >>
> > >
> >
> >
>