Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # user >> RE: Permission problem


+
Kevin Burton 2013-04-30, 14:55
+
Arpit Gupta 2013-04-30, 15:01
+
Kevin Burton 2013-04-30, 15:22
+
Arpit Gupta 2013-04-30, 15:47
+
Mohammad Tariq 2013-04-30, 17:06
+
Kevin Burton 2013-04-30, 13:36
Copy link to this message
-
Re: Permission problem
what is your mapred.system.dir set to in mapred-site.xml?

By default it will write to /tmp on hdfs.

So you can do the following

create /tmp on hdfs and chmod it to 777 as user hdfs and then restart jobtracker and tasktrackers.

In case its set to /mapred/something then create /mapred and chown it to user mapred.
--
Arpit Gupta
Hortonworks Inc.
http://hortonworks.com/

On Apr 30, 2013, at 6:36 AM, "Kevin Burton" <[EMAIL PROTECTED]> wrote:

> To further complicate the issue the log file in (/var/log/hadoop-0.20-mapreduce/hadoop-hadoop-jobtracker-devUbuntu05.log) is owned by mapred:mapred and the name of the file seems to indicate some other lineage (hadoop,hadoop). I am out of my league in understanding the permission structure for hadoop hdfs and mr. Ideas?
>  
> From: Kevin Burton [mailto:[EMAIL PROTECTED]]
> Sent: Tuesday, April 30, 2013 8:31 AM
> To: [EMAIL PROTECTED]
> Cc: 'Mohammad Tariq'
> Subject: RE: Permission problem
>  
> That is what I perceive as the problem. The hdfs file system was created with the user ‘hdfs’ owning the root (‘/’) but for some reason with a M/R job the user ‘mapred’ needs to have write permission to the root. I don’t know how to satisfy both conditions. That is one reason that I relaxed the permission to 775 so that the group would also have write permission but that didn’t seem to help.
>  
> From: Mohammad Tariq [mailto:[EMAIL PROTECTED]]
> Sent: Tuesday, April 30, 2013 8:20 AM
> To: Kevin Burton
> Subject: Re: Permission problem
>  
> user?"ls" shows "hdfs" and the log says "mapred"..
>
> Warm Regards,
> Tariq
> https://mtariq.jux.com/
> cloudfront.blogspot.com
>  
>
> On Tue, Apr 30, 2013 at 6:22 PM, Kevin Burton <[EMAIL PROTECTED]> wrote:
> I have relaxed it even further so now it is 775
>  
> kevin@devUbuntu05:/var/log/hadoop-0.20-mapreduce$ hadoop fs -ls -d /
> Found 1 items
> drwxrwxr-x   - hdfs supergroup          0 2013-04-29 15:43 /
>  
> But I still get this error:
>  
> 2013-04-30 07:43:02,520 FATAL org.apache.hadoop.mapred.JobTracker: org.apache.hadoop.security.AccessControlException: Permission denied: user=mapred, access=WRITE, inode="/":hdfs:supergroup:drwxrwxr-x
>  
>  
> From: Mohammad Tariq [mailto:[EMAIL PROTECTED]]
> Sent: Monday, April 29, 2013 5:10 PM
> To: [EMAIL PROTECTED]
> Subject: Re: Incompartible cluserIDS
>  
> make it 755.
>
> Warm Regards,
> Tariq
> https://mtariq.jux.com/
> cloudfront.blogspot.com
>  
>
>  

+
Kevin Burton 2013-04-30, 14:39
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB