Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> Re: multiusers in hadoop through LDAP


Copy link to this message
-
Re: multiusers in hadoop through LDAP
Hi
 Thanks a lot for your replies.

 I will try the LDAP+hadoop.security.group.mapping.ldap.*. Right now I can
not catch this question.
Regards.
2013/12/11 Jay Vyas <[EMAIL PROTECTED]>

> So, not knowing much about LDAP, but being very interested in the
> multiuser problem on multiuser filesystems, i was excited to see this
> question.... Im researching the same thing at the moment, and it seems
> obviated by the fact that :
>
> - the FileSystem API itslef provides implementations for getting group and
> user names / permissions....
>
> And furthermore
>
> - the linux task controllers launch jobs as the user submitting the job,
> whereas the regular task controllers launch tasksunder the YARN daemon
> name, iirc.
>
> So.... where does LDAP begin and TaskController / FileSystem notions of
> ownership end.... ?
>
> I guess I'm also asking what are the entites which are "ownable" in hadoop
> app , and how we can leverage the GroupMappingServiceProviders to deploy
> more flexible hadoop environments.
>
> Any thoughts on this would be appreciated.
>
> On Tue, Dec 10, 2013 at 6:38 PM, Adam Kawa <[EMAIL PROTECTED]> wrote:
>
>> Please have a look at hadoop.security.group.mapping.ldap.* settings as Hardik
>> Pandya suggests.
>>
>> ====>>
>> In advance, just to share our story related to LDAP +
>> hadoop.security.group.mapping.ldap.*, if you run into the same
>> limitation as we did:
>>
>> In many cases hadoop.security.group.mapping.ldap.* should solve your
>> problem. Unfortunately, they did now work for us. The problematic
>> setting relates to an additional filter to use when searching for LDAP
>> groups. We wanted to use posixGroups filter, but it is currently not
>> supported by Hadoop. Finally, we found a workaround using name service
>> switch configuration where we specified that the LDAP should the primary
>> source of information about groups of our users. This means that we solved
>> this problem on the operating system level, not on Hadoop level.
>>
>> You can read more about this issue here:
>>
>> http://hakunamapdata.com/a-user-having-surprising-troubles-running-more-resource-intensive-hive-queries/
>> and here
>> http://www.slideshare.net/AdamKawa/hadoop-adventures-at-spotify-strata-conference-hadoop-world-2013 (slides
>> 18-26).
>>
>>
>> 2013/12/10 Hardik Pandya <[EMAIL PROTECTED]>
>>
>>>
>>> have you looked at hadoop.security.group.mapping.ldap.* in
>>> hadoop-common/core-default.xml<http://hadoop.apache.org/docs/current2/hadoop-project-dist/hadoop-common/core-default.xml>
>>>
>>> additional resource<http://hakunamapdata.com/a-user-having-surprising-troubles-running-more-resource-intensive-hive-queries/>may help
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Tue, Dec 10, 2013 at 3:06 AM, YouPeng Yang <[EMAIL PROTECTED]
>>> > wrote:
>>>
>>>> Hi
>>>>
>>>>   In my cluster ,I want to have multiusers for different purpose.The
>>>> usual method is to add a user through the OS  on  Hadoop NameNode .
>>>>   I notice the hadoop also support to LDAP, could I add user through
>>>> LDAP instead through OS? So that if a user is authenticated by the LDAP
>>>> ,who will also access the HDFS directory?
>>>>
>>>>
>>>> Regards
>>>>
>>>
>>>
>>
>
>
> --
> Jay Vyas
> http://jayunit100.blogspot.com
>