Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Group mismatches?


Copy link to this message
-
Re: Group mismatches?
In all places I have found it only to be the primary group, not all
the users supplemental groups.

On Mon, Jul 16, 2012 at 3:05 PM, Clay B. <[EMAIL PROTECTED]> wrote:
> Hi all,
>
> I have a Hadoop cluster which uses Samba to map an Active Directory domain
> to my CentOS 5.7 Hadoop cluster. However, I notice a strange mismatch with
> groups. Does anyone have any debugging advice, or how to refresh the DFS
> groups mapping? If not, should I file a bug at
> https://issues.apache.org/jira/browse/HADOOP?
>
> # I see the following error:
> [clayb@hamster ~]$ hadoop fs -ls /projects/foobarcommander
> log4j:ERROR Could not find value for key log4j.appender.NullAppender
> log4j:ERROR Could not instantiate appender named "NullAppender".
> ls: could not get get listing for
> 'hdfs://hamster:9000/projects/foobarcommander' :
> org.apache.hadoop.security.AccessControlException: Permission denied:
> user=clayb, access=READ_EXECUTE,
> inode="/projects/foobarcommander":hadmin:foobarcommander:drwxrwx---
>
> # I verify group membership -- look a mismatch!
> [clayb@hamster ~]$ which groups
> /usr/bin/groups
> [clayb@hamster ~]$ groups
> foobarcommander xxx_rec_eng domain users all all_north america batchlogon
> xxx-s xxx03-s xxx1-admins xxx-emr-users xxx-emr-admins xxx1-users
> BUILTIN\users
> [clayb@hamster ~]$ hadoop dfsgroups
> log4j:ERROR Could not find value for key log4j.appender.NullAppender
> log4j:ERROR Could not instantiate appender named "NullAppender".
> clayb : domain users xxx_rec_eng xxx-emr-users all xxx-emr-admins batchlogon
> all_north america xxx1-users xxx-s xxx03-s xxx1-admins BUILTIN\users
>
> Notice, in particular the foobarcommander group is only shown for my
> /usr/bin/groups output. It looks like the following from the HDFS
> Permissions Guide[1] is not correct, in my case:
> "The group list is the equivalent of `bash -c groups`."
>
> # I have tried the following to no useful effect:
> [admin@hamster ~]$ hadoop dfsadmin -refreshUserToGroupsMappings
> log4j:ERROR Could not find value for key log4j.appender.NullAppender
> log4j:ERROR Could not instantiate appender named "NullAppender".
>
> # I do, however, see other users with the foobarcommander group, so the
> group should be "visible" to Hadoop:
> [clayb@hamster ~]$ hadoop dfsgroups pat
> log4j:ERROR Could not find value for key log4j.appender.NullAppender
> log4j:ERROR Could not instantiate appender named "NullAppender".
> pat : domain users all_north america all_san diego all foobarcommander
> BUILTIN\users
> # And 'hadoop mrgroups' (like dfsgroups) returns the same bad data, for me:
> [clayb@hamster ~]$ hadoop mrgroups
> log4j:ERROR Could not find value for key log4j.appender.NullAppender
> log4j:ERROR Could not instantiate appender named "NullAppender".
> clayb : domain users xxx_rec_eng xxx-emr-users all xxx-emr-admins batchlogon
> all_north america xxx1-users xxx-s xxx03-s xxx1-admins BUILTIN\users
>
> # And I see that the system is getting the right data via getent(1):
> [clayb@hamster ~]$ getent group foobarcommander
> foobarcommander:*:16777316:pat,user1,user2,user3,clayb,user4,user5,user6,user7,user8,user9,user10,user12,user13,user14,user15
>
> # I am using Cloudera's CDH3u4 Hadoop:
> [clayb@hamster ~]$ hadoop version
> Hadoop 0.20.2-cdh3u4
> Subversion file:///data/1/tmp/topdir/BUILD/hadoop-0.20.2-cdh3u4 -r
> 214dd731e3bdb687cb55988d3f47dd9e248c5690
> Compiled by root on Mon May  7 14:03:02 PDT 2012
> From source with checksum a60c9795e41a3248b212344fb131c12c
>
> I also do not see any obviously useful errors in my namenode logs.
>
> -Clay
>
> [1]:
> http://hadoop.apache.org/common/docs/r0.20.2/hdfs_permissions_guide.html#User+Identity
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB