Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # user - Security in Hadoop-1.0.0

Copy link to this message
Re: Security in Hadoop-1.0.0
Harsh J 2012-02-14, 13:44

What are you looking for, exactly? Are all you asking for is strong
authentication for your HDFS clusters such that no external user may
connect to it and read files (even those marked o+r)? If so, that is
what a HDFS security configuration, which we have pointed you to
already, aims to provide.

Know that LDAP isn't an "authentication" mechanism - and thats not
what you want to "integrate" HDFS with, for security. You need a
functional Kerberos environment that integrates with your LDAP, for
strong authentication of users (token based security). To setup
Kerberos integrated with your existing LDAP service, please follow
articles such as http://www.linux-mag.com/id/4738/

Once your Kerberos instance is setup to talk and authenticate users on
your LDAP instance, carry on with the guide pointed out earlier at
- which will essentially work for Apache Hadoop 1.x too. You only need
to bother with Kerberos after this point.

Hope this clears it up for you.

P.s. If your environment already uses Active Directory to manage
users, you can use that directly as well:

P.p.s. The doc page at
https://ccp.cloudera.com/display/CDHDOC/CDH3+Security+Guide carries
further articles on Kerberos and other security configs if you want to
read more - and all of the instructions would work with most upstream
releases too.

On Tue, Feb 14, 2012 at 1:48 PM, Stuti Awasthi <[EMAIL PROTECTED]> wrote:
> After some googling I found the following link :
> http://mapredit.blogspot.in/2011/10/secure-your-hadoop-cluster-part-i.html
> http://mapredit.blogspot.in/2011/10/secure-your-hadoop-cluster-part-ii.html
> But these mainly deals with applying LDAP for map-reduce. I want to configure LDAP for HDFS as well as mapreduce. Please suggest me some links through which I can configure dfs with LDP also.
> Thanks
> -----Original Message-----
> From: Stuti Awasthi
> Sent: Tuesday, February 14, 2012 12:28 PM
> Subject: RE: Security in Hadoop-1.0.0
> Thanks Patrick,
> The concept is clear to me now. As a first step I would like to configure LDAP with Hadoop.
> I am using Apache Hadoop 1.0.0 but not able to find configuration steps in this version documentation.
> It would be really helpful if someone can point me to relevant documentation of configuring this version of Hadoop with LDAP.
> Thanks
> From: Patrick Angeles [mailto:[EMAIL PROTECTED]]
> Sent: Monday, February 13, 2012 8:29 PM
> Subject: Re: Security in Hadoop-1.0.0
> LDAP and Kerberos are orthogonal in Hadoop, but both are often used together. LDAP allows for centralized user/group management (sort of like DNS for your users). Kerberos is for strong authentication of users.
> When using Kerberos in Hadoop, you want to propagate user/group identities to all your cluster nodes. (Otherwise, you might authenticate strongly, but your user ID doesn't exist in a Tasktracker so your job fails.) LDAP happens to be a common way to do this.
> Typically when you set up Kerberos, you also set up your cluster nodes to do LDAP authentication. You do this setup at the operating system level (via PAM).
> Note that you can also use Hue as your user-gateway to Hadoop. In this scenario, you can use an LDAP backend to authenticate users. You do not have to (but can) configure Hadoop with Kerberos.
> - P
> On Mon, Feb 13, 2012 at 3:11 AM, Stuti Awasthi <[EMAIL PROTECTED]> wrote:
> Hi,
> I am bit confused on Security part of Hadoop. Cluster is behind the firewall. I have read that Hadoop can be configured with LDAP also.
> I want to know which is better : configure Hadoop security with LDAP or Kerberos as both provide authentication.
> Please provide me more details on this as I am newbee in this part.
> Thanks
> -----Original Message-----
> From: alo alt [mailto:[EMAIL PROTECTED]]

Harsh J
Customer Ops. Engineer
Cloudera | http://tiny.cloudera.com/about