Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # dev >> ViewFileSystem default ugi on mount point


Copy link to this message
-
ViewFileSystem default ugi on mount point
Hi Devs,

ViewFileSystem by default tries to populate ugi information of mount point
based on the user who is running the command. For example if I execute
'hadoop fs -ls /' on '/' which happens to be a ViewFileSystem mountpoint,
then all mountpoint under '/' would who the permission and owner as 'lohit'

For example this check is coming from ViewFileSystem.java.

public FileStatus getFileStatus(Path f) throws IOException {
      checkPathIsSlash(f);
      return new FileStatus(0, true, 0, 0, creationTime, creationTime,
          PERMISSION_RRR, ugi.getUserName(), ugi.getGroupNames()[0],

          new Path(theInternalDir.fullPath).makeQualified(
              myUri, null));

In our clusters we do not create user accounts on worker nodes. Now if any
Map or Reduce code does getFileStatus on mountpoint they fail when it tries
to fetch ugi.getGroupNames()[0] with NPE. One option we are thinking is to
populate this with 'nobody':'nogroup' kind of string if we cannot resolve
to user/group name instead of failing with NPE. What do you guys think
about this approach. Any other ideas/suggestion?

--
Have a Nice Day!
Lohit