-Re: multiusers in hadoop through LDAP
Jay Vyas 2013-12-11, 02:41
So, not knowing much about LDAP, but being very interested in the multiuser
problem on multiuser filesystems, i was excited to see this question.... Im
researching the same thing at the moment, and it seems obviated by the fact
- the FileSystem API itslef provides implementations for getting group and
user names / permissions....
- the linux task controllers launch jobs as the user submitting the job,
whereas the regular task controllers launch tasksunder the YARN daemon
So.... where does LDAP begin and TaskController / FileSystem notions of
ownership end.... ?
I guess I'm also asking what are the entites which are "ownable" in hadoop
app , and how we can leverage the GroupMappingServiceProviders to deploy
more flexible hadoop environments.
Any thoughts on this would be appreciated.
On Tue, Dec 10, 2013 at 6:38 PM, Adam Kawa <[EMAIL PROTECTED]> wrote:
> Please have a look at hadoop.security.group.mapping.ldap.* settings as Hardik
> Pandya suggests.
> In advance, just to share our story related to LDAP +
> hadoop.security.group.mapping.ldap.*, if you run into the same limitation
> as we did:
> In many cases hadoop.security.group.mapping.ldap.* should solve your
> problem. Unfortunately, they did now work for us. The problematic setting
> relates to an additional filter to use when searching for LDAP groups. We
> wanted to use posixGroups filter, but it is currently not supported by
> Hadoop. Finally, we found a workaround using name service switch
> configuration where we specified that the LDAP should the primary source of
> information about groups of our users. This means that we solved this
> problem on the operating system level, not on Hadoop level.
> You can read more about this issue here:
> and here
> http://www.slideshare.net/AdamKawa/hadoop-adventures-at-spotify-strata-conference-hadoop-world-2013 (slides
> 2013/12/10 Hardik Pandya <[EMAIL PROTECTED]>
>> have you looked at hadoop.security.group.mapping.ldap.* in
>> additional resource<http://hakunamapdata.com/a-user-having-surprising-troubles-running-more-resource-intensive-hive-queries/>may help
>> On Tue, Dec 10, 2013 at 3:06 AM, YouPeng Yang <[EMAIL PROTECTED]>wrote:
>>> In my cluster ,I want to have multiusers for different purpose.The
>>> usual method is to add a user through the OS on Hadoop NameNode .
>>> I notice the hadoop also support to LDAP, could I add user through
>>> LDAP instead through OS? So that if a user is authenticated by the LDAP
>>> ,who will also access the HDFS directory?