Hi All,

We would like to integrate HBase with our corporate Active Directory for
authentication and authorization of HBase users.  I have gone through the
following links in HBase:
http://blog.cloudera.com/blog/2012/09/understanding-user-authentication-and-authorization-in-apache-hbase/
http://hbase.apache.org/book/security.html

I am a novice at Kerberos and Active Directory(LDAP). So, I would like to
know whether HBase can be integrated with Active Directory(LDAP) or not? Is
anyone using LDAP for Authentication and Authorization in HBase? If yes, it
would be great if someone can give me high level steps. Is it something
similar Kerberos integration with Active Directory in Hadoop?

--
Thanks & Regards,
Anil Gupta

Do you want to have just AD (via LDAP) based authentication (not sure
what I'm talking of here, really), or kerberos based authentication
but with an automatic binding to AD (via LDAP) for all the
allowed/available users?

On Tue, Dec 4, 2012 at 11:19 AM, anil gupta <[EMAIL PROTECTED]> wrote:
> Hi All,
>
> We would like to integrate HBase with our corporate Active Directory for
> authentication and authorization of HBase users.  I have gone through the
> following links in HBase:
> http://blog.cloudera.com/blog/2012/09/understanding-user-authentication-and-authorization-in-apache-hbase/
> http://hbase.apache.org/book/security.html
>
> I am a novice at Kerberos and Active Directory(LDAP). So, I would like to
> know whether HBase can be integrated with Active Directory(LDAP) or not? Is
> anyone using LDAP for Authentication and Authorization in HBase? If yes, it
> would be great if someone can give me high level steps. Is it something
> similar Kerberos integration with Active Directory in Hadoop?
>
> --
> Thanks & Regards,
> Anil Gupta

--
Harsh J

Hi Harsh,

Both of the approach you mentioned would be ok for us. We are aware that
Hadoop can be integrated with Active Directory. But, i could not find any
such reference for HBase. Do you have any idea about this? Any link or
documentation on this would be really helpful.

Thanks,
Anil Gupta

On Sat, Dec 8, 2012 at 7:54 AM, Harsh J <[EMAIL PROTECTED]> wrote:

> Do you want to have just AD (via LDAP) based authentication (not sure
> what I'm talking of here, really), or kerberos based authentication
> but with an automatic binding to AD (via LDAP) for all the
> allowed/available users?
>
> On Tue, Dec 4, 2012 at 11:19 AM, anil gupta <[EMAIL PROTECTED]> wrote:
> > Hi All,
> >
> > We would like to integrate HBase with our corporate Active Directory for
> > authentication and authorization of HBase users.  I have gone through the
> > following links in HBase:
> >
> http://blog.cloudera.com/blog/2012/09/understanding-user-authentication-and-authorization-in-apache-hbase/
> > http://hbase.apache.org/book/security.html
> >
> > I am a novice at Kerberos and Active Directory(LDAP). So, I would like to
> > know whether HBase can be integrated with Active Directory(LDAP) or not?
> Is
> > anyone using LDAP for Authentication and Authorization in HBase? If yes,
> it
> > would be great if someone can give me high level steps. Is it something
> > similar Kerberos integration with Active Directory in Hadoop?
> >
> > --
> > Thanks & Regards,
> > Anil Gupta
>
>
>
> --
> Harsh J
>

--
Thanks & Regards,
Anil Gupta

Hi,

An KDC can be made to trust an AD, which would solve your need. This
https://ccp.cloudera.com/display/CDH4DOC/Integrating+Hadoop+Security+with+Active+Directory
is one guide that details on how to set it up.

HBase wraps very little logic over Hadoop's security providing
classes, so proper Hadoop security configuration (such as
auth_to_local rules, etc.) will work for HBase directly and you can
have all your AD users onboard for authentication.

Does this answer your question?

On Sat, Dec 8, 2012 at 11:43 PM, anil gupta <[EMAIL PROTECTED]> wrote:
> Hi Harsh,
>
> Both of the approach you mentioned would be ok for us. We are aware that
> Hadoop can be integrated with Active Directory. But, i could not find any
> such reference for HBase. Do you have any idea about this? Any link or
> documentation on this would be really helpful.
>
> Thanks,
> Anil Gupta
>
> On Sat, Dec 8, 2012 at 7:54 AM, Harsh J <[EMAIL PROTECTED]> wrote:
>
>> Do you want to have just AD (via LDAP) based authentication (not sure
>> what I'm talking of here, really), or kerberos based authentication
>> but with an automatic binding to AD (via LDAP) for all the
>> allowed/available users?
>>
>> On Tue, Dec 4, 2012 at 11:19 AM, anil gupta <[EMAIL PROTECTED]> wrote:
>> > Hi All,
>> >
>> > We would like to integrate HBase with our corporate Active Directory for
>> > authentication and authorization of HBase users.  I have gone through the
>> > following links in HBase:
>> >
>> http://blog.cloudera.com/blog/2012/09/understanding-user-authentication-and-authorization-in-apache-hbase/
>> > http://hbase.apache.org/book/security.html
>> >
>> > I am a novice at Kerberos and Active Directory(LDAP). So, I would like to
>> > know whether HBase can be integrated with Active Directory(LDAP) or not?
>> Is
>> > anyone using LDAP for Authentication and Authorization in HBase? If yes,
>> it
>> > would be great if someone can give me high level steps. Is it something
>> > similar Kerberos integration with Active Directory in Hadoop?
>> >
>> > --
>> > Thanks & Regards,
>> > Anil Gupta
>>
>>
>>
>> --
>> Harsh J
>>
>
>
>
> --
> Thanks & Regards,
> Anil Gupta

--
Harsh J

Hi Harsh,

We are in process of installing a HBase cluster with a secure HDFS and
HBase. We already have a secure HDFS integrated with AD but we are still
trying to figure out a way to integrate HBase with AD(directly or
indirectly throgh KDC). I think my colleague has already implemented the
stuff provided in previous link for securing HDFS. :) However, i will try
to correlate this article for HBase installation and see if we can make
HBase work with AD. Thanks a lot for your response and time.

PS: It might be possible to integrate HBase with AD but till now i have
found no reference or documentation for it.

Thanks,
Anil Gupta

On Sat, Dec 8, 2012 at 11:17 AM, Harsh J <[EMAIL PROTECTED]> wrote:

> Hi,
>
> An KDC can be made to trust an AD, which would solve your need. This
>
> https://ccp.cloudera.com/display/CDH4DOC/Integrating+Hadoop+Security+with+Active+Directory
> is one guide that details on how to set it up.
>
> HBase wraps very little logic over Hadoop's security providing
> classes, so proper Hadoop security configuration (such as
> auth_to_local rules, etc.) will work for HBase directly and you can
> have all your AD users onboard for authentication.
>
> Does this answer your question?
>
> On Sat, Dec 8, 2012 at 11:43 PM, anil gupta <[EMAIL PROTECTED]> wrote:
> > Hi Harsh,
> >
> > Both of the approach you mentioned would be ok for us. We are aware that
> > Hadoop can be integrated with Active Directory. But, i could not find any
> > such reference for HBase. Do you have any idea about this? Any link or
> > documentation on this would be really helpful.
> >
> > Thanks,
> > Anil Gupta
> >
> > On Sat, Dec 8, 2012 at 7:54 AM, Harsh J <[EMAIL PROTECTED]> wrote:
> >
> >> Do you want to have just AD (via LDAP) based authentication (not sure
> >> what I'm talking of here, really), or kerberos based authentication
> >> but with an automatic binding to AD (via LDAP) for all the
> >> allowed/available users?
> >>
> >> On Tue, Dec 4, 2012 at 11:19 AM, anil gupta <[EMAIL PROTECTED]>
> wrote:
> >> > Hi All,
> >> >
> >> > We would like to integrate HBase with our corporate Active Directory
> for
> >> > authentication and authorization of HBase users.  I have gone through
> the
> >> > following links in HBase:
> >> >
> >>
> http://blog.cloudera.com/blog/2012/09/understanding-user-authentication-and-authorization-in-apache-hbase/
> >> > http://hbase.apache.org/book/security.html
> >> >
> >> > I am a novice at Kerberos and Active Directory(LDAP). So, I would
> like to
> >> > know whether HBase can be integrated with Active Directory(LDAP) or
> not?
> >> Is
> >> > anyone using LDAP for Authentication and Authorization in HBase? If
> yes,
> >> it
> >> > would be great if someone can give me high level steps. Is it
> something
> >> > similar Kerberos integration with Active Directory in Hadoop?
> >> >
> >> > --
> >> > Thanks & Regards,
> >> > Anil Gupta
> >>
> >>
> >>
> >> --
> >> Harsh J
> >>
> >
> >
> >
> > --
> > Thanks & Regards,
> > Anil Gupta
>
>
>
> --
> Harsh J
>

--
Thanks & Regards,
Anil Gupta

Hi,

Correct me if I'm wrong, but HBase presently has no reliance on the
concept of groups, just users. For authenticating users, it relies on
Hadoop Common's security libraries, which is the same as is used by
HDFS for authentication. The Hadoop Common security libraries provided
auth_to_local form of configs for transforming AD->KDC principal
names, which HBase can leverage as well (via the same configs).

Essentially, if you make HBase see Hadoop's proper security configs
(including any AD-required ones), then that's all there is to it.

Back to the concept of groups, the reason I mentioned it is that for
permissions model the NameNode uses a Groups mapping plugin, to get an
accurate picture of the groups a user may belong to. For this to be
consistent in an AD environment, Hadoop Common provides a LDAP-mapping
feature. This lies outside of authentication layers, and is useful
only in cases of HDFS and MapReduce which have group-wise applications
and configurations.

On Mon, Dec 10, 2012 at 2:20 AM, anil gupta <[EMAIL PROTECTED]> wrote:
> Hi Harsh,
>
> We are in process of installing a HBase cluster with a secure HDFS and
> HBase. We already have a secure HDFS integrated with AD but we are still
> trying to figure out a way to integrate HBase with AD(directly or
> indirectly throgh KDC). I think my colleague has already implemented the
> stuff provided in previous link for securing HDFS. :) However, i will try
> to correlate this article for HBase installation and see if we can make
> HBase work with AD. Thanks a lot for your response and time.
>
> PS: It might be possible to integrate HBase with AD but till now i have
> found no reference or documentation for it.
>
> Thanks,
> Anil Gupta
>
> On Sat, Dec 8, 2012 at 11:17 AM, Harsh J <[EMAIL PROTECTED]> wrote:
>
>> Hi,
>>
>> An KDC can be made to trust an AD, which would solve your need. This
>>
>> https://ccp.cloudera.com/display/CDH4DOC/Integrating+Hadoop+Security+with+Active+Directory
>> is one guide that details on how to set it up.
>>
>> HBase wraps very little logic over Hadoop's security providing
>> classes, so proper Hadoop security configuration (such as
>> auth_to_local rules, etc.) will work for HBase directly and you can
>> have all your AD users onboard for authentication.
>>
>> Does this answer your question?
>>
>> On Sat, Dec 8, 2012 at 11:43 PM, anil gupta <[EMAIL PROTECTED]> wrote:
>> > Hi Harsh,
>> >
>> > Both of the approach you mentioned would be ok for us. We are aware that
>> > Hadoop can be integrated with Active Directory. But, i could not find any
>> > such reference for HBase. Do you have any idea about this? Any link or
>> > documentation on this would be really helpful.
>> >
>> > Thanks,
>> > Anil Gupta
>> >
>> > On Sat, Dec 8, 2012 at 7:54 AM, Harsh J <[EMAIL PROTECTED]> wrote:
>> >
>> >> Do you want to have just AD (via LDAP) based authentication (not sure
>> >> what I'm talking of here, really), or kerberos based authentication
>> >> but with an automatic binding to AD (via LDAP) for all the
>> >> allowed/available users?
>> >>
>> >> On Tue, Dec 4, 2012 at 11:19 AM, anil gupta <[EMAIL PROTECTED]>
>> wrote:
>> >> > Hi All,
>> >> >
>> >> > We would like to integrate HBase with our corporate Active Directory
>> for
>> >> > authentication and authorization of HBase users.  I have gone through
>> the
>> >> > following links in HBase:
>> >> >
>> >>
>> http://blog.cloudera.com/blog/2012/09/understanding-user-authentication-and-authorization-in-apache-hbase/
>> >> > http://hbase.apache.org/book/security.html
>> >> >
>> >> > I am a novice at Kerberos and Active Directory(LDAP). So, I would
>> like to
>> >> > know whether HBase can be integrated with Active Directory(LDAP) or
>> not?
>> >> Is
>> >> > anyone using LDAP for Authentication and Authorization in HBase? If
>> yes,
>> >> it
>> >> > would be great if someone can give me high level steps. Is it
>> something
>> >> > similar Kerberos integration with Active Directory in Hadoop?
>> >> >

Harsh J

Hi Harsh,

HBase has a concept of ACL. But, these ACL's are maintained as another
system table "*_acl_*"(similar to Meta and Root) in HBase.  See:
hbase.apache.org/book/hbase.accesscontrol.configuration.html.
Instead of HBase maintaining these ACL's as a system table we want HBase to
understand the ACL's of AD(directly or indirectly through Kerberos) so that
we are not maintaining users at many places.
So, for a client to query a HBase table, first the client will need to
authenticate through HBase Client API.(For example: client authenticates to
Oracle through JDBC api before a query is run on the DB and this Oracle
instance is integrated to AD). I hope this clarifies my requirement.

Thanks,
Anil Gupta
On Sun, Dec 9, 2012 at 12:58 PM, Harsh J <[EMAIL PROTECTED]> wrote:

> Hi,
>
> Correct me if I'm wrong, but HBase presently has no reliance on the
> concept of groups, just users. For authenticating users, it relies on
> Hadoop Common's security libraries, which is the same as is used by
> HDFS for authentication. The Hadoop Common security libraries provided
> auth_to_local form of configs for transforming AD->KDC principal
> names, which HBase can leverage as well (via the same configs).
>
> Essentially, if you make HBase see Hadoop's proper security configs
> (including any AD-required ones), then that's all there is to it.
>
> Back to the concept of groups, the reason I mentioned it is that for
> permissions model the NameNode uses a Groups mapping plugin, to get an
> accurate picture of the groups a user may belong to. For this to be
> consistent in an AD environment, Hadoop Common provides a LDAP-mapping
> feature. This lies outside of authentication layers, and is useful
> only in cases of HDFS and MapReduce which have group-wise applications
> and configurations.
>
> On Mon, Dec 10, 2012 at 2:20 AM, anil gupta <[EMAIL PROTECTED]> wrote:
> > Hi Harsh,
> >
> > We are in process of installing a HBase cluster with a secure HDFS and
> > HBase. We already have a secure HDFS integrated with AD but we are still
> > trying to figure out a way to integrate HBase with AD(directly or
> > indirectly throgh KDC). I think my colleague has already implemented the
> > stuff provided in previous link for securing HDFS. :) However, i will try
> > to correlate this article for HBase installation and see if we can make
> > HBase work with AD. Thanks a lot for your response and time.
> >
> > PS: It might be possible to integrate HBase with AD but till now i have
> > found no reference or documentation for it.
> >
> > Thanks,
> > Anil Gupta
> >
> > On Sat, Dec 8, 2012 at 11:17 AM, Harsh J <[EMAIL PROTECTED]> wrote:
> >
> >> Hi,
> >>
> >> An KDC can be made to trust an AD, which would solve your need. This
> >>
> >>
> https://ccp.cloudera.com/display/CDH4DOC/Integrating+Hadoop+Security+with+Active+Directory
> >> is one guide that details on how to set it up.
> >>
> >> HBase wraps very little logic over Hadoop's security providing
> >> classes, so proper Hadoop security configuration (such as
> >> auth_to_local rules, etc.) will work for HBase directly and you can
> >> have all your AD users onboard for authentication.
> >>
> >> Does this answer your question?
> >>
> >> On Sat, Dec 8, 2012 at 11:43 PM, anil gupta <[EMAIL PROTECTED]>
> wrote:
> >> > Hi Harsh,
> >> >
> >> > Both of the approach you mentioned would be ok for us. We are aware
> that
> >> > Hadoop can be integrated with Active Directory. But, i could not find
> any
> >> > such reference for HBase. Do you have any idea about this? Any link or
> >> > documentation on this would be really helpful.
> >> >
> >> > Thanks,
> >> > Anil Gupta
> >> >
> >> > On Sat, Dec 8, 2012 at 7:54 AM, Harsh J <[EMAIL PROTECTED]> wrote:
> >> >
> >> >> Do you want to have just AD (via LDAP) based authentication (not sure
> >> >> what I'm talking of here, really), or kerberos based authentication
> >> >> but with an automatic binding to AD (via LDAP) for all the
> >> >> allowed/available users?

Thanks & Regards,
Anil Gupta

Ah alright. To rephrase my answer: Authentication in HBase via AD is
supported, but direct Authorization of tables via AD is not.

You'd need to either come up with your own co-processors or enhance
the AccessController to feed its ACL data off of LDAP instead of a
system table (a pluggable design perhaps, or if going the cheap way, a
continuous application that syncs the LDAP ACLs state to the HBase
system table state periodically).

On Mon, Dec 10, 2012 at 3:17 AM, anil gupta <[EMAIL PROTECTED]> wrote:
> Hi Harsh,
>
> HBase has a concept of ACL. But, these ACL's are maintained as another
> system table "*_acl_*"(similar to Meta and Root) in HBase.  See:
> hbase.apache.org/book/hbase.accesscontrol.configuration.html.
> Instead of HBase maintaining these ACL's as a system table we want HBase to
> understand the ACL's of AD(directly or indirectly through Kerberos) so that
> we are not maintaining users at many places.
> So, for a client to query a HBase table, first the client will need to
> authenticate through HBase Client API.(For example: client authenticates to
> Oracle through JDBC api before a query is run on the DB and this Oracle
> instance is integrated to AD). I hope this clarifies my requirement.
>
> Thanks,
> Anil Gupta
>
>
> On Sun, Dec 9, 2012 at 12:58 PM, Harsh J <[EMAIL PROTECTED]> wrote:
>
>> Hi,
>>
>> Correct me if I'm wrong, but HBase presently has no reliance on the
>> concept of groups, just users. For authenticating users, it relies on
>> Hadoop Common's security libraries, which is the same as is used by
>> HDFS for authentication. The Hadoop Common security libraries provided
>> auth_to_local form of configs for transforming AD->KDC principal
>> names, which HBase can leverage as well (via the same configs).
>>
>> Essentially, if you make HBase see Hadoop's proper security configs
>> (including any AD-required ones), then that's all there is to it.
>>
>> Back to the concept of groups, the reason I mentioned it is that for
>> permissions model the NameNode uses a Groups mapping plugin, to get an
>> accurate picture of the groups a user may belong to. For this to be
>> consistent in an AD environment, Hadoop Common provides a LDAP-mapping
>> feature. This lies outside of authentication layers, and is useful
>> only in cases of HDFS and MapReduce which have group-wise applications
>> and configurations.
>>
>> On Mon, Dec 10, 2012 at 2:20 AM, anil gupta <[EMAIL PROTECTED]> wrote:
>> > Hi Harsh,
>> >
>> > We are in process of installing a HBase cluster with a secure HDFS and
>> > HBase. We already have a secure HDFS integrated with AD but we are still
>> > trying to figure out a way to integrate HBase with AD(directly or
>> > indirectly throgh KDC). I think my colleague has already implemented the
>> > stuff provided in previous link for securing HDFS. :) However, i will try
>> > to correlate this article for HBase installation and see if we can make
>> > HBase work with AD. Thanks a lot for your response and time.
>> >
>> > PS: It might be possible to integrate HBase with AD but till now i have
>> > found no reference or documentation for it.
>> >
>> > Thanks,
>> > Anil Gupta
>> >
>> > On Sat, Dec 8, 2012 at 11:17 AM, Harsh J <[EMAIL PROTECTED]> wrote:
>> >
>> >> Hi,
>> >>
>> >> An KDC can be made to trust an AD, which would solve your need. This
>> >>
>> >>
>> https://ccp.cloudera.com/display/CDH4DOC/Integrating+Hadoop+Security+with+Active+Directory
>> >> is one guide that details on how to set it up.
>> >>
>> >> HBase wraps very little logic over Hadoop's security providing
>> >> classes, so proper Hadoop security configuration (such as
>> >> auth_to_local rules, etc.) will work for HBase directly and you can
>> >> have all your AD users onboard for authentication.
>> >>
>> >> Does this answer your question?
>> >>
>> >> On Sat, Dec 8, 2012 at 11:43 PM, anil gupta <[EMAIL PROTECTED]>
>> wrote:
>> >> > Hi Harsh,
>> >> >
>> >> > Both of the approach you mentioned would be ok for us. We are aware

Harsh J

Thanks a lot for the responses, Harsh. Even i thought that if HBase does
not supports Authorization with AD then we might need to write a
coprocessor.

On Sun, Dec 9, 2012 at 2:33 PM, Harsh J <[EMAIL PROTECTED]> wrote:

> Ah alright. To rephrase my answer: Authentication in HBase via AD is
> supported, but direct Authorization of tables via AD is not.
>
> You'd need to either come up with your own co-processors or enhance
> the AccessController to feed its ACL data off of LDAP instead of a
> system table (a pluggable design perhaps, or if going the cheap way, a
> continuous application that syncs the LDAP ACLs state to the HBase
> system table state periodically).
>
> On Mon, Dec 10, 2012 at 3:17 AM, anil gupta <[EMAIL PROTECTED]> wrote:
> > Hi Harsh,
> >
> > HBase has a concept of ACL. But, these ACL's are maintained as another
> > system table "*_acl_*"(similar to Meta and Root) in HBase.  See:
> > hbase.apache.org/book/hbase.accesscontrol.configuration.html.
> > Instead of HBase maintaining these ACL's as a system table we want HBase
> to
> > understand the ACL's of AD(directly or indirectly through Kerberos) so
> that
> > we are not maintaining users at many places.
> > So, for a client to query a HBase table, first the client will need to
> > authenticate through HBase Client API.(For example: client authenticates
> to
> > Oracle through JDBC api before a query is run on the DB and this Oracle
> > instance is integrated to AD). I hope this clarifies my requirement.
> >
> > Thanks,
> > Anil Gupta
> >
> >
> > On Sun, Dec 9, 2012 at 12:58 PM, Harsh J <[EMAIL PROTECTED]> wrote:
> >
> >> Hi,
> >>
> >> Correct me if I'm wrong, but HBase presently has no reliance on the
> >> concept of groups, just users. For authenticating users, it relies on
> >> Hadoop Common's security libraries, which is the same as is used by
> >> HDFS for authentication. The Hadoop Common security libraries provided
> >> auth_to_local form of configs for transforming AD->KDC principal
> >> names, which HBase can leverage as well (via the same configs).
> >>
> >> Essentially, if you make HBase see Hadoop's proper security configs
> >> (including any AD-required ones), then that's all there is to it.
> >>
> >> Back to the concept of groups, the reason I mentioned it is that for
> >> permissions model the NameNode uses a Groups mapping plugin, to get an
> >> accurate picture of the groups a user may belong to. For this to be
> >> consistent in an AD environment, Hadoop Common provides a LDAP-mapping
> >> feature. This lies outside of authentication layers, and is useful
> >> only in cases of HDFS and MapReduce which have group-wise applications
> >> and configurations.
> >>
> >> On Mon, Dec 10, 2012 at 2:20 AM, anil gupta <[EMAIL PROTECTED]>
> wrote:
> >> > Hi Harsh,
> >> >
> >> > We are in process of installing a HBase cluster with a secure HDFS and
> >> > HBase. We already have a secure HDFS integrated with AD but we are
> still
> >> > trying to figure out a way to integrate HBase with AD(directly or
> >> > indirectly throgh KDC). I think my colleague has already implemented
> the
> >> > stuff provided in previous link for securing HDFS. :) However, i will
> try
> >> > to correlate this article for HBase installation and see if we can
> make
> >> > HBase work with AD. Thanks a lot for your response and time.
> >> >
> >> > PS: It might be possible to integrate HBase with AD but till now i
> have
> >> > found no reference or documentation for it.
> >> >
> >> > Thanks,
> >> > Anil Gupta
> >> >
> >> > On Sat, Dec 8, 2012 at 11:17 AM, Harsh J <[EMAIL PROTECTED]> wrote:
> >> >
> >> >> Hi,
> >> >>
> >> >> An KDC can be made to trust an AD, which would solve your need. This
> >> >>
> >> >>
> >>
> https://ccp.cloudera.com/display/CDH4DOC/Integrating+Hadoop+Security+with+Active+Directory
> >> >> is one guide that details on how to set it up.
> >> >>
> >> >> HBase wraps very little logic over Hadoop's security providing
> >> >> classes, so proper Hadoop security configuration (such as

Thanks & Regards,
Anil Gupta