I will try to investigate the issue with fair scheduler. Do let us know if switching to default or the capacity scheduler solved the issue.
On Oct 10, 2012, at 9:32 AM, "Goldstone, Robin J." <[EMAIL PROTECTED]> wrote:
> There is no /hadoop1 directory. It is //hadoop1 which is the name of the server running the name node daemon:
> Per offline conversations with Arpit, it appears this problem is related to the fact that I am using the fair scheduler. The fair scheduler is designed to run map reduce jobs as the user, rather than under the mapred username. Apparently there are some issues with this scheduler related to permissions on certain directories not allowing other users to execute/write in places that are necessary for the job to run. I haven't yet tried Arpit's suggestion to switch to the task scheduler but I imagine it will resolve my issue, at least for now. Ultimately I do want to use the fair scheduler, as multi-tenancy is a key requirement for our Hadoop deployment.
> From: Manu S <[EMAIL PROTECTED]>
> Reply-To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
> Date: Wednesday, October 10, 2012 3:34 AM
> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
> Subject: Re: issue with permissions of mapred.system.dir
> What is the permission for /hadoop1 dir in HDFS? Is "mapred" user have permission on the same directory?
> Manu S
> On Wed, Oct 10, 2012 at 5:52 AM, Arpit Gupta <[EMAIL PROTECTED]> wrote:
>> what is your "mapreduce.jobtracker.staging.root.dir" set to. This is a directory that needs to be writable by the user and is is recommended to be set to "/user" so it writes in appropriate users home directory.
>> Arpit Gupta
>> Hortonworks Inc.
>> On Oct 9, 2012, at 4:44 PM, "Goldstone, Robin J." <[EMAIL PROTECTED]> wrote:
>>> I am bringing up a Hadoop cluster for the first time (but am an experienced sysadmin with lots of cluster experience) and running into an issue with permissions on mapred.system.dir. It has generally been a chore to figure out all the various directories that need to be created to get Hadoop working, some on the local FS, others within HDFS, getting the right ownership and permissions, etc.. I think I am mostly there but can't seem to get past my current issue with mapred.system.dir.
>>> Some general info first:
>>> OS: RHEL6
>>> Hadoop version: hadoop-1.0.3-1.x86_64
>>> 20 node cluster configured as follows
>>> 1 node as primary namenode
>>> 1 node as secondary namenode + job tracker
>>> 18 nodes as datanode + tasktracker
>>> I have HDFS up and running and have the following in mapred-site.xml:
>>> <description>Shared data for JT - this must be in HDFS</description>
>>> I have created this directory in HDFS, owner mapred:hadoop, permissions 700 which seems to be the most common recommendation amongst multiple, often conflicting articles about how to set up Hadoop. Here is the top level of my filesystem:
>>> hyperion-hdp4@hdfs:hadoop fs -ls /
>>> Found 3 items
>>> drwx------ - mapred hadoop 0 2012-10-09 12:58 /mapred
>>> drwxrwxrwx - hdfs hadoop 0 2012-10-09 13:00 /tmp
>>> drwxr-xr-x - hdfs hadoop 0 2012-10-09 12:51 /user
>>> Note, it doesn't seem to really matter what permissions I set on /mapred since when the Jobtracker starts up it changes them to 700.
>>> However, when I try to run the hadoop example teragen program as a "regular" user I am getting this error:
>>> hyperion-hdp4@robing:hadoop jar /usr/share/hadoop/hadoop-examples*.jar teragen -D dfs.block.size=536870912 10000000000 /user/robing/terasort-input
>>> Generating 10000000000 using 2 maps with step of 5000000000
>>> 12/10/09 16:27:02 INFO mapred.JobClient: Running job: job_201210072045_0003