Hadoop, mail # user - Re: Problem setting up Hadoop security with active directory using one-way cross-realm configuration

Problem setting up Hadoop security with active directory using one-way cross-realm configuration

Ivan Frain 2012-07-25, 09:29

*Hi all,*
*
*
*I am trying to setup a one-way cross realm trust between a MIT KDC and an
active directory server and up to now I did not success.*
*I hope someone in this list will be able to help me.*
*
*
*My config is as follows:*
* - hadoop version: 0.23.1 with security enable (kerberos).*
* - hadoop realm (mitkdc): HADOOP.REALM*
* - 1 linux node (mitkdc.hadoop.realm - 192.168.198.254) running : hdfs
namenode, hdfs datanode, mit kdc*
* - 1 windows node (ad.domain.realm - 192.168.198.253) running: active
directory 2003*
* - AD realm: DOMAIN.REALM*
*
*
*Everything works well with kerberos enabled if I only use the linux
machine with users having principal in the mitkdc: [EMAIL PROTECTED]M*
*
*
*What I am trying to do is to use the user database in the Active directory
(users with principals like [EMAIL PROTECTED]M)*
*
*
*To do that, I setup a one-way cross realm as explained here:
https://ccp.cloudera.com/display/CDH4DOC/Integrating+Hadoop+Security+with+Active+Directory
*
*
*
*From the linux machine I can authenticate against an active directory user
with the kinit command but when I perform a query using the hadoop command
I have the following error message:*
---------------------
hdfs@mitkdc:~$ kinit [EMAIL PROTECTED]M
Password for [EMAIL PROTECTED]M:

hdfs@mitkdc:~$ klist -e
Ticket cache: FILE:/tmp/krb5cc_10003
Default principal: [EMAIL PROTECTED]M

Valid starting Expires Service principal
25/07/2012 11:00 25/07/2012 20:59 krbtgt/[EMAIL PROTECTED]M
renew until 26/07/2012 11:00, Etype (skey, tkt): arcfour-hmac, arcfour-hmac

hdfs@mitkdc:~$ hadoop/bin/hadoop fs -ls /user
12/07/25 11:00:50 ERROR security.UserGroupInformation:
PriviledgedActionException as:[EMAIL PROTECTED]M (auth:KERBEROS)
cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by
GSSException: No valid credentials provided (Mechanism level: Fail to
create credential. (63) - No service creds)]
12/07/25 11:00:50 INFO security.UserGroupInformation: Initiating logout for
[EMAIL PROTECTED]M
12/07/25 11:00:50 INFO security.UserGroupInformation: Initiating re-login
for [EMAIL PROTECTED]M
12/07/25 11:00:53 ERROR security.UserGroupInformation:
PriviledgedActionException as:[EMAIL PROTECTED]M (auth:KERBEROS)
cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by
GSSException: No valid credentials provided (Mechanism level: Fail to
create credential. (63) - No service creds)]
12/07/25 11:00:53 WARN security.UserGroupInformation: Not attempting to
re-login since the last re-login was attempted less than 600 seconds before.
12/07/25 11:00:56 ERROR security.UserGroupInformation:
PriviledgedActionException as:[EMAIL PROTECTED]M (auth:KERBEROS)
cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by
GSSException: No valid credentials provided (Mechanism level: Fail to
create credential. (63) - No service creds)]
12/07/25 11:00:56 WARN security.UserGroupInformation: Not attempting to
re-login since the last re-login was attempted less than 600 seconds before.
12/07/25 11:00:58 ERROR security.UserGroupInformation:
PriviledgedActionException as:[EMAIL PROTECTED]M (auth:KERBEROS)
cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by
GSSException: No valid credentials provided (Mechanism level: Fail to
create credential. (63) - No service creds)]
12/07/25 11:00:58 WARN security.UserGroupInformation: Not attempting to
re-login since the last re-login was attempted less than 600 seconds before.
12/07/25 11:00:59 ERROR security.UserGroupInformation:
PriviledgedActionException as:[EMAIL PROTECTED]M (auth:KERBEROS)
cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by
GSSException: No valid credentials provided (Mechanism level: Fail to
create credential. (63) - No service creds)]
12/07/25 11:00:59 WARN security.UserGroupInformation: Not attempting to
re-login since the last re-login was attempted less than 600 seconds before.
12/07/25 11:01:02 ERROR security.UserGroupInformation:
PriviledgedActionException as:[EMAIL PROTECTED]M (auth:KERBEROS)
cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by
GSSException: No valid credentials provided (Mechanism level: Fail to
create credential. (63) - No service creds)]
12/07/25 11:01:02 WARN ipc.Client: Couldn't setup connection for
[EMAIL PROTECTED]M to hdfs/[EMAIL PROTECTED]M
12/07/25 11:01:02 ERROR security.UserGroupInformation:
PriviledgedActionException as:[EMAIL PROTECTED]M (auth:KERBEROS)
cause:java.io.IOException: Couldn't setup connection for
[EMAIL PROTECTED]Mto hdfs/[EMAIL PROTECTED]M
ls: Failed on local exception: java.io.IOException: Couldn't setup
connection for [EMAIL PROTECTED]M to hdfs/[EMAIL PROTECTED]M;
Host Details : local host is: "mitkdc.hadoop.realm/192.168.198.254";
destination host is: ""mitkdc.hadoop.realm":8020;

*On the mitkdc server log I can see something like the following meaning
that encoded types are not supported: *

Jul 25 09:53:33 mitkdc.hadoop.realm krb5kdc[8630](info): TGS_REQ (5 etypes
{3 1 23 16 17}) 192.168.198.254: PROCESS_TGS: authtime 0, <unknown client>
for <unknown server>, No matching key in entry having a permitted enctype
Jul 25 09:53:36 mitkdc.hadoop.realm krb5kdc[8630](info): TGS_REQ (5 etypes
{3 1 23 16 17}) 192.168.198.254: PROCESS_TGS: authtime 0, <unknown client>
for <unknown server>, No matching key in entry having a permitted enctype
Jul 25 09:53:37 mitkdc.hadoop.realm krb5kdc[8630](info): TGS_REQ (5 etypes
{3 1 23 16 17}) 192.168.198.254: PROCESS_TGS: authtime 0, <unknown client>
for <unknown server>, No matching key in entry having a permitted enctype
Jul 25 09:53:37 mitkdc.hadoop.realm krb5kdc[8630](info): TGS_REQ (5 etypes
{3 1 23 16 17}) 192.168.198.254: PROCESS_TGS: authtime 0, <unknown client>
for <unknown server>, No matching key in entry having a permitted enctype
Jul 25 09:54:25 mitkdc.hadoop.realm krb5kdc[8630](info): TGS_REQ (5 etypes
{3 1 23 16 17}) 192.168.198.254: PROCESS_TGS: a

Re: Problem setting up Hadoop security with active directory using one-way cross-realm configuration

Mapred Learn 2012-07-25, 14:59

You need to set up a local realm on your KDC ( linux) and run commands on windows AD to add this realm as a trust realm on your AD realm.

After this you need to modify your /etc/krb5.conf to include this local realm as trust realm to your AD realm.

And then you should be all set.

Sent from my iPhone

On Jul 25, 2012, at 2:29 AM, Ivan Frain <[EMAIL PROTECTED]> wrote:

> *Hi all,*
> *
> *
> *I am trying to setup a one-way cross realm trust between a MIT KDC and an
> active directory server and up to now I did not success.*
> *I hope someone in this list will be able to help me.*
> *
> *
> *My config is as follows:*
> * - hadoop version: 0.23.1 with security enable (kerberos).*
> * - hadoop realm (mitkdc): HADOOP.REALM*
> * - 1 linux node (mitkdc.hadoop.realm - 192.168.198.254) running : hdfs
> namenode, hdfs datanode, mit kdc*
> * - 1 windows node (ad.domain.realm - 192.168.198.253) running: active
> directory 2003*
> * - AD realm: DOMAIN.REALM*
> *
> *
> *Everything works well with kerberos enabled if I only use the linux
> machine with users having principal in the mitkdc: [EMAIL PROTECTED]M*
> *
> *
> *What I am trying to do is to use the user database in the Active directory
> (users with principals like [EMAIL PROTECTED]M)*
> *
> *
> *To do that, I setup a one-way cross realm as explained here:
> https://ccp.cloudera.com/display/CDH4DOC/Integrating+Hadoop+Security+with+Active+Directory
> *
> *
> *
> *From the linux machine I can authenticate against an active directory user
> with the kinit command but when I perform a query using the hadoop command
> I have the following error message:*
> ---------------------
> hdfs@mitkdc:~$ kinit [EMAIL PROTECTED]M
> Password for [EMAIL PROTECTED]M:
>
> hdfs@mitkdc:~$ klist -e
> Ticket cache: FILE:/tmp/krb5cc_10003
> Default principal: [EMAIL PROTECTED]M
>
> Valid starting Expires Service principal
> 25/07/2012 11:00 25/07/2012 20:59 krbtgt/[EMAIL PROTECTED]M
> renew until 26/07/2012 11:00, Etype (skey, tkt): arcfour-hmac, arcfour-hmac
>
> hdfs@mitkdc:~$ hadoop/bin/hadoop fs -ls /user
> 12/07/25 11:00:50 ERROR security.UserGroupInformation:
> PriviledgedActionException as:[EMAIL PROTECTED]M (auth:KERBEROS)
> cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Fail to
> create credential. (63) - No service creds)]
> 12/07/25 11:00:50 INFO security.UserGroupInformation: Initiating logout for
> [EMAIL PROTECTED]M
> 12/07/25 11:00:50 INFO security.UserGroupInformation: Initiating re-login
> for [EMAIL PROTECTED]M
> 12/07/25 11:00:53 ERROR security.UserGroupInformation:
> PriviledgedActionException as:[EMAIL PROTECTED]M (auth:KERBEROS)
> cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Fail to
> create credential. (63) - No service creds)]
> 12/07/25 11:00:53 WARN security.UserGroupInformation: Not attempting to
> re-login since the last re-login was attempted less than 600 seconds before.
> 12/07/25 11:00:56 ERROR security.UserGroupInformation:
> PriviledgedActionException as:[EMAIL PROTECTED]M (auth:KERBEROS)
> cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Fail to
> create credential. (63) - No service creds)]
> 12/07/25 11:00:56 WARN security.UserGroupInformation: Not attempting to
> re-login since the last re-login was attempted less than 600 seconds before.
> 12/07/25 11:00:58 ERROR security.UserGroupInformation:
> PriviledgedActionException as:[EMAIL PROTECTED]M (auth:KERBEROS)
> cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Fail to
> create credential. (63) - No service creds)]
> 12/07/25 11:00:58 WARN security.UserGroupInformation: Not attempting to
> re-login since the last re-login was attempted less than 600 seconds before.

Re: Problem setting up Hadoop security with active directory using one-way cross-realm configuration

Ivan Frain 2012-07-25, 15:27

Thanks for your answer.

I think I already did what you propose. Some comments in the remaining.
2012/7/25 Mapred Learn <[EMAIL PROTECTED]>

> You need to set up a local realm on your KDC ( linux) and run commands on
> windows AD to add this realm as a trust realm on your AD realm.
>

I set up a KDC on the linux machine and configure a one-way incoming trust
on AD to be trusted by the local KDC. I set the enc type as well on AD. I
also create the appropriate remote TGT on the local KDC:
krbtgt/[EMAIL PROTECTED]M with the same encoding type
>
> After this you need to modify your /etc/krb5.conf to include this local
> realm as trust realm to your AD realm.
>

Here is the /etc/krb5.conf located in my local kdc on mitkdc.hadoop.realm
machine. May be something is wrong there:

[libdefaults]
   default_realm = HADOOP.REALM
default_tkt_enctypes = arcfour-hmac-md5
default_tgs_enctypes = arcfour-hmac-md5

[realms]
   HADOOP.REALM = {
   kdc = mitkdc.hadoop.realm
   admin_server = mitkdc.hadoop.realm
default_domain = hadoop.realm
   }
DOMAIN.REALM = {
kdc = ad.domain.realm
admin_server = ad.domain.realm
default_domain = domain.realm
}

[domain_realm]
.hadoop.realm = HADOOP.REALM
hadoop.realm = HADOOP.REALM
.domain.realm = DOMAIN.REALM
domain.realm = DOMAIN.REALM

>
> And then you should be all set.
>
>
I was hoping so but it is not ... yet ... the case

> Sent from my iPhone
>
> On Jul 25, 2012, at 2:29 AM, Ivan Frain <[EMAIL PROTECTED]> wrote:
>
> > *Hi all,*
> > *
> > *
> > *I am trying to setup a one-way cross realm trust between a MIT KDC and
> an
> > active directory server and up to now I did not success.*
> > *I hope someone in this list will be able to help me.*
> > *
> > *
> > *My config is as follows:*
> > * - hadoop version: 0.23.1 with security enable (kerberos).*
> > * - hadoop realm (mitkdc): HADOOP.REALM*
> > * - 1 linux node (mitkdc.hadoop.realm - 192.168.198.254) running : hdfs
> > namenode, hdfs datanode, mit kdc*
> > * - 1 windows node (ad.domain.realm - 192.168.198.253) running: active
> > directory 2003*
> > * - AD realm: DOMAIN.REALM*
> > *
> > *
> > *Everything works well with kerberos enabled if I only use the linux
> > machine with users having principal in the mitkdc: [EMAIL PROTECTED]M*
> > *
> > *
> > *What I am trying to do is to use the user database in the Active
> directory
> > (users with principals like [EMAIL PROTECTED]M)*
> > *
> > *
> > *To do that, I setup a one-way cross realm as explained here:
> >
> https://ccp.cloudera.com/display/CDH4DOC/Integrating+Hadoop+Security+with+Active+Directory
> > *
> > *
> > *
> > *From the linux machine I can authenticate against an active directory
> user
> > with the kinit command but when I perform a query using the hadoop
> command
> > I have the following error message:*
> > ---------------------
> > hdfs@mitkdc:~$ kinit [EMAIL PROTECTED]M
> > Password for [EMAIL PROTECTED]M:
> >
> > hdfs@mitkdc:~$ klist -e
> > Ticket cache: FILE:/tmp/krb5cc_10003
> > Default principal: [EMAIL PROTECTED]M
> >
> > Valid starting Expires Service principal
> > 25/07/2012 11:00 25/07/2012 20:59 krbtgt/[EMAIL PROTECTED]M
> > renew until 26/07/2012 11:00, Etype (skey, tkt): arcfour-hmac,
> arcfour-hmac
> >
> > hdfs@mitkdc:~$ hadoop/bin/hadoop fs -ls /user
> > 12/07/25 11:00:50 ERROR security.UserGroupInformation:
> > PriviledgedActionException as:[EMAIL PROTECTED]M (auth:KERBEROS)
> > cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by
> > GSSException: No valid credentials provided (Mechanism level: Fail to
> > create credential. (63) - No service creds)]
> > 12/07/25 11:00:50 INFO security.UserGroupInformation: Initiating logout
> for
> > [EMAIL PROTECTED]M
> > 12/07/25 11:00:50 INFO security.UserGroupInformation: Initiating re-login
> > for [EMAIL PROTECTED]M
> > 12/07/25 11:00:53 ERROR security.UserGroupInformation:
> > PriviledgedActionException as:[EMAIL PROTECTED]M (auth:KERBEROS)
> > cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by

Ivan Frain
11, route de Grenade
31530 Saint-Paul-sur-Save
mobile: +33 (0)6 52 52 47 07

Re: Problem setting up Hadoop security with active directory using one-way cross-realm configuration

Mapred Learn 2012-07-25, 16:11

Krb5 looks good.
Can you also share commands you ran in your Windows AD ?

Sent from my iPhone

On Jul 25, 2012, at 8:27 AM, Ivan Frain <[EMAIL PROTECTED]> wrote:

> Thanks for your answer.
>
> I think I already did what you propose. Some comments in the remaining.
>
>
> 2012/7/25 Mapred Learn <[EMAIL PROTECTED]>
>
>> You need to set up a local realm on your KDC ( linux) and run commands on
>> windows AD to add this realm as a trust realm on your AD realm.
>>
>
> I set up a KDC on the linux machine and configure a one-way incoming trust
> on AD to be trusted by the local KDC. I set the enc type as well on AD. I
> also create the appropriate remote TGT on the local KDC:
> krbtgt/[EMAIL PROTECTED]M with the same encoding type
>
>
>>
>> After this you need to modify your /etc/krb5.conf to include this local
>> realm as trust realm to your AD realm.
>>
>
> Here is the /etc/krb5.conf located in my local kdc on mitkdc.hadoop.realm
> machine. May be something is wrong there:
>
> [libdefaults]
> default_realm = HADOOP.REALM
> default_tkt_enctypes = arcfour-hmac-md5
> default_tgs_enctypes = arcfour-hmac-md5
>
> [realms]
> HADOOP.REALM = {
> kdc = mitkdc.hadoop.realm
> admin_server = mitkdc.hadoop.realm
> default_domain = hadoop.realm
> }
> DOMAIN.REALM = {
> kdc = ad.domain.realm
> admin_server = ad.domain.realm
> default_domain = domain.realm
> }
>
> [domain_realm]
> .hadoop.realm = HADOOP.REALM
> hadoop.realm = HADOOP.REALM
> .domain.realm = DOMAIN.REALM
> domain.realm = DOMAIN.REALM
>
>
>
>>
>> And then you should be all set.
>>
>>
> I was hoping so but it is not ... yet ... the case
>
>
>
>> Sent from my iPhone
>>
>> On Jul 25, 2012, at 2:29 AM, Ivan Frain <[EMAIL PROTECTED]> wrote:
>>
>>> *Hi all,*
>>> *
>>> *
>>> *I am trying to setup a one-way cross realm trust between a MIT KDC and
>> an
>>> active directory server and up to now I did not success.*
>>> *I hope someone in this list will be able to help me.*
>>> *
>>> *
>>> *My config is as follows:*
>>> * - hadoop version: 0.23.1 with security enable (kerberos).*
>>> * - hadoop realm (mitkdc): HADOOP.REALM*
>>> * - 1 linux node (mitkdc.hadoop.realm - 192.168.198.254) running : hdfs
>>> namenode, hdfs datanode, mit kdc*
>>> * - 1 windows node (ad.domain.realm - 192.168.198.253) running: active
>>> directory 2003*
>>> * - AD realm: DOMAIN.REALM*
>>> *
>>> *
>>> *Everything works well with kerberos enabled if I only use the linux
>>> machine with users having principal in the mitkdc: [EMAIL PROTECTED]M*
>>> *
>>> *
>>> *What I am trying to do is to use the user database in the Active
>> directory
>>> (users with principals like [EMAIL PROTECTED]M)*
>>> *
>>> *
>>> *To do that, I setup a one-way cross realm as explained here:
>>>
>> https://ccp.cloudera.com/display/CDH4DOC/Integrating+Hadoop+Security+with+Active+Directory
>>> *
>>> *
>>> *
>>> *From the linux machine I can authenticate against an active directory
>> user
>>> with the kinit command but when I perform a query using the hadoop
>> command
>>> I have the following error message:*
>>> ---------------------
>>> hdfs@mitkdc:~$ kinit [EMAIL PROTECTED]M
>>> Password for [EMAIL PROTECTED]M:
>>>
>>> hdfs@mitkdc:~$ klist -e
>>> Ticket cache: FILE:/tmp/krb5cc_10003
>>> Default principal: [EMAIL PROTECTED]M
>>>
>>> Valid starting Expires Service principal
>>> 25/07/2012 11:00 25/07/2012 20:59 krbtgt/[EMAIL PROTECTED]M
>>> renew until 26/07/2012 11:00, Etype (skey, tkt): arcfour-hmac,
>> arcfour-hmac
>>>
>>> hdfs@mitkdc:~$ hadoop/bin/hadoop fs -ls /user
>>> 12/07/25 11:00:50 ERROR security.UserGroupInformation:
>>> PriviledgedActionException as:[EMAIL PROTECTED]M (auth:KERBEROS)
>>> cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by
>>> GSSException: No valid credentials provided (Mechanism level: Fail to
>>> create credential. (63) - No service creds)]
>>> 12/07/25 11:00:50 INFO security.UserGroupInformation: Initiating logout

Re: Problem setting up Hadoop security with active directory using one-way cross-realm configuration

Ivan Frain 2012-07-25, 16:25

In AD:
- I have created a one way incoming trust using the GUI (I guess it is the
equivalent of the "netdom trust").
- ksetup /addkdc HADOOP.REALM mitkdc.hadoop.realm
- ksetup /SetEncTypeAttr HADOOP.REALM RC4-HMAC-MD5

What do you think ?
2012/7/25 Mapred Learn <[EMAIL PROTECTED]>

> Krb5 looks good.
> Can you also share commands you ran in your Windows AD ?
>
> Sent from my iPhone
>
> On Jul 25, 2012, at 8:27 AM, Ivan Frain <[EMAIL PROTECTED]> wrote:
>
> > Thanks for your answer.
> >
> > I think I already did what you propose. Some comments in the remaining.
> >
> >
> > 2012/7/25 Mapred Learn <[EMAIL PROTECTED]>
> >
> >> You need to set up a local realm on your KDC ( linux) and run commands
> on
> >> windows AD to add this realm as a trust realm on your AD realm.
> >>
> >
> > I set up a KDC on the linux machine and configure a one-way incoming
> trust
> > on AD to be trusted by the local KDC. I set the enc type as well on AD. I
> > also create the appropriate remote TGT on the local KDC:
> > krbtgt/[EMAIL PROTECTED]M with the same encoding type
> >
> >
> >>
> >> After this you need to modify your /etc/krb5.conf to include this local
> >> realm as trust realm to your AD realm.
> >>
> >
> > Here is the /etc/krb5.conf located in my local kdc on mitkdc.hadoop.realm
> > machine. May be something is wrong there:
> >
> > [libdefaults]
> > default_realm = HADOOP.REALM
> > default_tkt_enctypes = arcfour-hmac-md5
> > default_tgs_enctypes = arcfour-hmac-md5
> >
> > [realms]
> > HADOOP.REALM = {
> > kdc = mitkdc.hadoop.realm
> > admin_server = mitkdc.hadoop.realm
> > default_domain = hadoop.realm
> > }
> > DOMAIN.REALM = {
> > kdc = ad.domain.realm
> > admin_server = ad.domain.realm
> > default_domain = domain.realm
> > }
> >
> > [domain_realm]
> > .hadoop.realm = HADOOP.REALM
> > hadoop.realm = HADOOP.REALM
> > .domain.realm = DOMAIN.REALM
> > domain.realm = DOMAIN.REALM
> >
> >
> >
> >>
> >> And then you should be all set.
> >>
> >>
> > I was hoping so but it is not ... yet ... the case
> >
> >
> >
> >> Sent from my iPhone
> >>
> >> On Jul 25, 2012, at 2:29 AM, Ivan Frain <[EMAIL PROTECTED]> wrote:
> >>
> >>> *Hi all,*
> >>> *
> >>> *
> >>> *I am trying to setup a one-way cross realm trust between a MIT KDC and
> >> an
> >>> active directory server and up to now I did not success.*
> >>> *I hope someone in this list will be able to help me.*
> >>> *
> >>> *
> >>> *My config is as follows:*
> >>> * - hadoop version: 0.23.1 with security enable (kerberos).*
> >>> * - hadoop realm (mitkdc): HADOOP.REALM*
> >>> * - 1 linux node (mitkdc.hadoop.realm - 192.168.198.254) running :
> hdfs
> >>> namenode, hdfs datanode, mit kdc*
> >>> * - 1 windows node (ad.domain.realm - 192.168.198.253) running: active
> >>> directory 2003*
> >>> * - AD realm: DOMAIN.REALM*
> >>> *
> >>> *
> >>> *Everything works well with kerberos enabled if I only use the linux
> >>> machine with users having principal in the mitkdc: [EMAIL PROTECTED]M*
> >>> *
> >>> *
> >>> *What I am trying to do is to use the user database in the Active
> >> directory
> >>> (users with principals like [EMAIL PROTECTED]M)*
> >>> *
> >>> *
> >>> *To do that, I setup a one-way cross realm as explained here:
> >>>
> >>
> https://ccp.cloudera.com/display/CDH4DOC/Integrating+Hadoop+Security+with+Active+Directory
> >>> *
> >>> *
> >>> *
> >>> *From the linux machine I can authenticate against an active directory
> >> user
> >>> with the kinit command but when I perform a query using the hadoop
> >> command
> >>> I have the following error message:*
> >>> ---------------------
> >>> hdfs@mitkdc:~$ kinit [EMAIL PROTECTED]M
> >>> Password for [EMAIL PROTECTED]M:
> >>>
> >>> hdfs@mitkdc:~$ klist -e
> >>> Ticket cache: FILE:/tmp/krb5cc_10003
> >>> Default principal: [EMAIL PROTECTED]M
> >>>
> >>> Valid starting Expires Service principal
> >>> 25/07/2012 11:00 25/07/2012 20:59 krbtgt/[EMAIL PROTECTED]M
> >>> renew until 26/07/2012 11:00, Etype (skey, tkt): arcfour-hmac,

Ivan Frain
11, route de Grenade
31530 Saint-Paul-sur-Save
mobile: +33 (0)6 52 52 47 07

Re: Problem setting up Hadoop security with active directory using one-way cross-realm configuration

Mapred Learn 2012-07-25, 19:07

You have to run netdom trust command to set up trust realm on your windows
AD after ksetup command:

netdom trust HADOOP.REALM /Domain:DOMAIN.REALM /add /realm
/passwordT:<Password>

and then ktpass command:

ktpass /MITRealmName HADOOP.REALM /TrustEncryp RC4

I think the second ksetup command you ran is not needed.

Only first one and the two above are usually sufficient.

Let me know how does it go after these 2 commands.

On Wed, Jul 25, 2012 at 9:25 AM, Ivan Frain <[EMAIL PROTECTED]> wrote:

> In AD:
> - I have created a one way incoming trust using the GUI (I guess it is the
> equivalent of the "netdom trust").
> - ksetup /addkdc HADOOP.REALM mitkdc.hadoop.realm
> - ksetup /SetEncTypeAttr HADOOP.REALM RC4-HMAC-MD5
>
> What do you think ?
>
>
>
>
> 2012/7/25 Mapred Learn <[EMAIL PROTECTED]>
>
> > Krb5 looks good.
> > Can you also share commands you ran in your Windows AD ?
> >
> > Sent from my iPhone
> >
> > On Jul 25, 2012, at 8:27 AM, Ivan Frain <[EMAIL PROTECTED]> wrote:
> >
> > > Thanks for your answer.
> > >
> > > I think I already did what you propose. Some comments in the remaining.
> > >
> > >
> > > 2012/7/25 Mapred Learn <[EMAIL PROTECTED]>
> > >
> > >> You need to set up a local realm on your KDC ( linux) and run commands
> > on
> > >> windows AD to add this realm as a trust realm on your AD realm.
> > >>
> > >
> > > I set up a KDC on the linux machine and configure a one-way incoming
> > trust
> > > on AD to be trusted by the local KDC. I set the enc type as well on
> AD. I
> > > also create the appropriate remote TGT on the local KDC:
> > > krbtgt/[EMAIL PROTECTED]M with the same encoding type
> > >
> > >
> > >>
> > >> After this you need to modify your /etc/krb5.conf to include this
> local
> > >> realm as trust realm to your AD realm.
> > >>
> > >
> > > Here is the /etc/krb5.conf located in my local kdc on
> mitkdc.hadoop.realm
> > > machine. May be something is wrong there:
> > >
> > > [libdefaults]
> > > default_realm = HADOOP.REALM
> > > default_tkt_enctypes = arcfour-hmac-md5
> > > default_tgs_enctypes = arcfour-hmac-md5
> > >
> > > [realms]
> > > HADOOP.REALM = {
> > > kdc = mitkdc.hadoop.realm
> > > admin_server = mitkdc.hadoop.realm
> > > default_domain = hadoop.realm
> > > }
> > > DOMAIN.REALM = {
> > > kdc = ad.domain.realm
> > > admin_server = ad.domain.realm
> > > default_domain = domain.realm
> > > }
> > >
> > > [domain_realm]
> > > .hadoop.realm = HADOOP.REALM
> > > hadoop.realm = HADOOP.REALM
> > > .domain.realm = DOMAIN.REALM
> > > domain.realm = DOMAIN.REALM
> > >
> > >
> > >
> > >>
> > >> And then you should be all set.
> > >>
> > >>
> > > I was hoping so but it is not ... yet ... the case
> > >
> > >
> > >
> > >> Sent from my iPhone
> > >>
> > >> On Jul 25, 2012, at 2:29 AM, Ivan Frain <[EMAIL PROTECTED]> wrote:
> > >>
> > >>> *Hi all,*
> > >>> *
> > >>> *
> > >>> *I am trying to setup a one-way cross realm trust between a MIT KDC
> and
> > >> an
> > >>> active directory server and up to now I did not success.*
> > >>> *I hope someone in this list will be able to help me.*
> > >>> *
> > >>> *
> > >>> *My config is as follows:*
> > >>> * - hadoop version: 0.23.1 with security enable (kerberos).*
> > >>> * - hadoop realm (mitkdc): HADOOP.REALM*
> > >>> * - 1 linux node (mitkdc.hadoop.realm - 192.168.198.254) running :
> > hdfs
> > >>> namenode, hdfs datanode, mit kdc*
> > >>> * - 1 windows node (ad.domain.realm - 192.168.198.253) running:
> active
> > >>> directory 2003*
> > >>> * - AD realm: DOMAIN.REALM*
> > >>> *
> > >>> *
> > >>> *Everything works well with kerberos enabled if I only use the linux
> > >>> machine with users having principal in the mitkdc: [EMAIL PROTECTED]M
> *
> > >>> *
> > >>> *
> > >>> *What I am trying to do is to use the user database in the Active
> > >> directory
> > >>> (users with principals like [EMAIL PROTECTED]M)*
> > >>> *
> > >>> *
> > >>> *To do that, I setup a one-way cross realm as explained here:
> > >>

RE: Problem setting up Hadoop security with active directory using one-way cross-realm configuration

Guillaume Polaert 2012-10-15, 10:08

Hi Ivan,

Did you solve your problem ?
I've the same issue. I can run Hadoop commands after a kinit with a "local principal" (@CLUSTER.HADOOP.DEV) but it doesn't work with AD user (@AD.HADOOP.DEV).

Could you help me ?
Thanks

Guillaume Polaert | Cyrès Conseil
-----Message d'origine-----
De : Ivan Frain [mailto:[EMAIL PROTECTED]]
Envoyé : mercredi 25 juillet 2012 18:25
À : [EMAIL PROTECTED]
Objet : Re: Problem setting up Hadoop security with active directory using one-way cross-realm configuration

In AD:
- I have created a one way incoming trust using the GUI (I guess it is the equivalent of the "netdom trust").
- ksetup /addkdc HADOOP.REALM mitkdc.hadoop.realm
- ksetup /SetEncTypeAttr HADOOP.REALM RC4-HMAC-MD5

What do you think ?
2012/7/25 Mapred Learn <[EMAIL PROTECTED]>

> Krb5 looks good.
> Can you also share commands you ran in your Windows AD ?
>
> Sent from my iPhone
>
> On Jul 25, 2012, at 8:27 AM, Ivan Frain <[EMAIL PROTECTED]> wrote:
>
> > Thanks for your answer.
> >
> > I think I already did what you propose. Some comments in the remaining.
> >
> >
> > 2012/7/25 Mapred Learn <[EMAIL PROTECTED]>
> >
> >> You need to set up a local realm on your KDC ( linux) and run
> >> commands
> on
> >> windows AD to add this realm as a trust realm on your AD realm.
> >>
> >
> > I set up a KDC on the linux machine and configure a one-way
> > incoming
> trust
> > on AD to be trusted by the local KDC. I set the enc type as well on
> > AD. I also create the appropriate remote TGT on the local KDC:
> > krbtgt/[EMAIL PROTECTED]M with the same encoding type
> >
> >
> >>
> >> After this you need to modify your /etc/krb5.conf to include this
> >> local realm as trust realm to your AD realm.
> >>
> >
> > Here is the /etc/krb5.conf located in my local kdc on
> > mitkdc.hadoop.realm machine. May be something is wrong there:
> >
> > [libdefaults]
> > default_realm = HADOOP.REALM
> > default_tkt_enctypes = arcfour-hmac-md5 default_tgs_enctypes =
> > arcfour-hmac-md5
> >
> > [realms]
> > HADOOP.REALM = {
> > kdc = mitkdc.hadoop.realm
> > admin_server = mitkdc.hadoop.realm default_domain =
> > hadoop.realm
> > }
> > DOMAIN.REALM = {
> > kdc = ad.domain.realm
> > admin_server = ad.domain.realm
> > default_domain = domain.realm
> > }
> >
> > [domain_realm]
> > .hadoop.realm = HADOOP.REALM
> > hadoop.realm = HADOOP.REALM
> > .domain.realm = DOMAIN.REALM
> > domain.realm = DOMAIN.REALM
> >
> >
> >
> >>
> >> And then you should be all set.
> >>
> >>
> > I was hoping so but it is not ... yet ... the case
> >
> >
> >
> >> Sent from my iPhone
> >>
> >> On Jul 25, 2012, at 2:29 AM, Ivan Frain <[EMAIL PROTECTED]> wrote:
> >>
> >>> *Hi all,*
> >>> *
> >>> *
> >>> *I am trying to setup a one-way cross realm trust between a MIT
> >>> KDC and
> >> an
> >>> active directory server and up to now I did not success.* *I hope
> >>> someone in this list will be able to help me.*
> >>> *
> >>> *
> >>> *My config is as follows:*
> >>> * - hadoop version: 0.23.1 with security enable (kerberos).*
> >>> * - hadoop realm (mitkdc): HADOOP.REALM*
> >>> * - 1 linux node (mitkdc.hadoop.realm - 192.168.198.254) running :
> hdfs
> >>> namenode, hdfs datanode, mit kdc*
> >>> * - 1 windows node (ad.domain.realm - 192.168.198.253) running:
> >>> active directory 2003*
> >>> * - AD realm: DOMAIN.REALM*
> >>> *
> >>> *
> >>> *Everything works well with kerberos enabled if I only use the
> >>> linux machine with users having principal in the mitkdc:
> >>> [EMAIL PROTECTED]M*
> >>> *
> >>> *
> >>> *What I am trying to do is to use the user database in the Active
> >> directory
> >>> (users with principals like [EMAIL PROTECTED]M)*
> >>> *
> >>> *
> >>> *To do that, I setup a one-way cross realm as explained here:
> >>>
> >>
> https://ccp.cloudera.com/display/CDH4DOC/Integrating+Hadoop+Security+w
> ith+Active+Directory
> >>> *
> >>> *
> >>> *
> >>> *From the linux machine I can authenticate against an active

Ivan Frain
11, route de Grenade
31530 Saint-Paul-sur-Save
mobile: +33 (0)6 52 52 47 07

RE: Problem setting up Hadoop security with active directory using one-way cross-realm configuration

Guillaume Polaert 2012-10-15, 13:16

It's working.
I haven't configured the property <name>hadoop.security.auth_to_local</name> for AD REALM.

Guillaume Polaert | Cyrès Conseil
-----Message d'origine-----
De : Guillaume Polaert [mailto:[EMAIL PROTECTED]]
Envoyé : lundi 15 octobre 2012 12:08
À : [EMAIL PROTECTED]; [EMAIL PROTECTED]
Objet : RE: Problem setting up Hadoop security with active directory using one-way cross-realm configuration

Hi Ivan,

Did you solve your problem ?
I've the same issue. I can run Hadoop commands after a kinit with a "local principal" (@CLUSTER.HADOOP.DEV) but it doesn't work with AD user (@AD.HADOOP.DEV).

Could you help me ?
Thanks

Guillaume Polaert | Cyrès Conseil
-----Message d'origine-----
De : Ivan Frain [mailto:[EMAIL PROTECTED]] Envoyé : mercredi 25 juillet 2012 18:25 À : [EMAIL PROTECTED] Objet : Re: Problem setting up Hadoop security with active directory using one-way cross-realm configuration

In AD:
- I have created a one way incoming trust using the GUI (I guess it is the equivalent of the "netdom trust").
- ksetup /addkdc HADOOP.REALM mitkdc.hadoop.realm
- ksetup /SetEncTypeAttr HADOOP.REALM RC4-HMAC-MD5

What do you think ?
2012/7/25 Mapred Learn <[EMAIL PROTECTED]>

> Krb5 looks good.
> Can you also share commands you ran in your Windows AD ?
>
> Sent from my iPhone
>
> On Jul 25, 2012, at 8:27 AM, Ivan Frain <[EMAIL PROTECTED]> wrote:
>
> > Thanks for your answer.
> >
> > I think I already did what you propose. Some comments in the remaining.
> >
> >
> > 2012/7/25 Mapred Learn <[EMAIL PROTECTED]>
> >
> >> You need to set up a local realm on your KDC ( linux) and run
> >> commands
> on
> >> windows AD to add this realm as a trust realm on your AD realm.
> >>
> >
> > I set up a KDC on the linux machine and configure a one-way
> > incoming
> trust
> > on AD to be trusted by the local KDC. I set the enc type as well on
> > AD. I also create the appropriate remote TGT on the local KDC:
> > krbtgt/[EMAIL PROTECTED]M with the same encoding type
> >
> >
> >>
> >> After this you need to modify your /etc/krb5.conf to include this
> >> local realm as trust realm to your AD realm.
> >>
> >
> > Here is the /etc/krb5.conf located in my local kdc on
> > mitkdc.hadoop.realm machine. May be something is wrong there:
> >
> > [libdefaults]
> > default_realm = HADOOP.REALM
> > default_tkt_enctypes = arcfour-hmac-md5 default_tgs_enctypes > > arcfour-hmac-md5
> >
> > [realms]
> > HADOOP.REALM = {
> > kdc = mitkdc.hadoop.realm
> > admin_server = mitkdc.hadoop.realm default_domain =
> > hadoop.realm
> > }
> > DOMAIN.REALM = {
> > kdc = ad.domain.realm
> > admin_server = ad.domain.realm
> > default_domain = domain.realm
> > }
> >
> > [domain_realm]
> > .hadoop.realm = HADOOP.REALM
> > hadoop.realm = HADOOP.REALM
> > .domain.realm = DOMAIN.REALM
> > domain.realm = DOMAIN.REALM
> >
> >
> >
> >>
> >> And then you should be all set.
> >>
> >>
> > I was hoping so but it is not ... yet ... the case
> >
> >
> >
> >> Sent from my iPhone
> >>
> >> On Jul 25, 2012, at 2:29 AM, Ivan Frain <[EMAIL PROTECTED]> wrote:
> >>
> >>> *Hi all,*
> >>> *
> >>> *
> >>> *I am trying to setup a one-way cross realm trust between a MIT
> >>> KDC and
> >> an
> >>> active directory server and up to now I did not success.* *I hope
> >>> someone in this list will be able to help me.*
> >>> *
> >>> *
> >>> *My config is as follows:*
> >>> * - hadoop version: 0.23.1 with security enable (kerberos).*
> >>> * - hadoop realm (mitkdc): HADOOP.REALM*
> >>> * - 1 linux node (mitkdc.hadoop.realm - 192.168.198.254) running :
> hdfs
> >>> namenode, hdfs datanode, mit kdc*
> >>> * - 1 windows node (ad.domain.realm - 192.168.198.253) running:
> >>> active directory 2003*
> >>> * - AD realm: DOMAIN.REALM*
> >>> *
> >>> *
> >>> *Everything works well with kerberos enabled if I only use the
> >>> linux machine with users having principal in the mitkdc:
> >>> [EMAIL PROTECTED]M*
> >>> *
> >>> *

Ivan Frain
11, route de Grenade
31530 Saint-Paul-sur-Save
mobile: +33 (0)6 52 52 47 07