Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Fair scheduler.


Copy link to this message
-
Re: Fair scheduler.
Harsh.. i am testing it again according to your last instruction.

>> 2. Define your required queues:
>>mapred.job.queues set to "default,foo,bar" for example, for 3 queues:
>>default, foo and bar.

>From http://archive.cloudera.com/cdh/3/hadoop-0.20.2-cdh3u4/cluster_setup.html#Configuring+the+Environment+of+the+Hadoop+Daemons
I couldn't find "mapred.job.queues" from that link so i have been
using mapred.queue.names which might be the case that it is my fault.

Please suggest

On Wed, Oct 17, 2012 at 8:43 AM, Harsh J <[EMAIL PROTECTED]> wrote:
> Hey Robin,
>
> Thanks for the detailed post.
>
> Just looked at your older thread, and you're right, the JT does write
> into its system dir for users' job info and token files when
> initializing the Job. The bug you ran into and the exception+trace you
> got makes sense now.
>
> I just didn't see it on version which Patai seems to be using. I think
> if he specifies a proper staging directory, he'll go through, cause
> his trace is different than that of MAPREDUCE-4398 (i.e. system dir
> vs. staging dir - you had system dir unfortunately).
>
> On Wed, Oct 17, 2012 at 8:39 PM, Goldstone, Robin J.
> <[EMAIL PROTECTED]> wrote:
>> Yes, you would think that users shouldn't need to write to
>> mapred.system.dir, yet that seems to be the case.  I posted details about
>> my configuration along with full stack traces last week.  I won't re-post
>> everything but essentially I have mapred.system.dir defined as a directory
>> in HDFS owned by mapred:hadoop.  I initially set the permissions to 755
>> but when the job tracker started up it changed the permissions to 700.
>> Then when I ran a job as a regular user I got this error:
>>
>> 12/10/09 16:27:03 INFO mapred.JobClient: Job Failed: Job initialization
>> failed:
>> org.apache.hadoop.security.AccessControlException:
>> org.apache.hadoop.security.AccessControlException: Permission denied:
>> user=robing, access=EXECUTE, inode="mapred":mapred:hadoop:rwx------
>>
>>
>> I then manually changed the permissions back to 755 and ran again and got
>> this error:
>> 12/10/09 16:31:30 INFO mapred.JobClient: Job Failed: Job initialization
>> failed:
>> org.apache.hadoop.security.AccessControlException:
>> org.apache.hadoop.security.AccessControlException: Permission denied:
>> user=robing, access=WRITE, inode="mapred":mapred:hadoop:rwxr-xr-x
>>
>> I then changed the permissions to 777 and the job ran successfully.  This
>> suggests that some process was trying to write to write to
>> mapred.system.dir but did not have sufficient permissions.  The
>> speculation is that this was being attempted under my uid instead of
>> mapred.  Perhaps it is something else. I welcome your suggestions.
>>
>>
>> For completeness, I also have mapred.jobtracker.staging.root.dir set to
>> /user within HDFS.  I can verify the staging files are going there but
>> something else is still trying to access mapred.system.dir.
>>
>> Robin Goldstone, LLNL
>>
>> On 10/17/12 12:00 AM, "Harsh J" <[EMAIL PROTECTED]> wrote:
>>
>>>Hi,
>>>
>>>Regular users never write into the mapred.system.dir AFAICT. That
>>>directory, is just for the JT to use to mark its presence and to
>>>"expose" the distributed filesystem it will be relying on.
>>>
>>>Users write to their respective staging directories, which lies
>>>elsewhere and is per-user.
>>>
>>>Let me post my environment:
>>>
>>>- mapred.system.dir (A HDFS Dir for a JT to register itself) set to
>>>"/tmp/mapred/system". The /tmp/mapred and /tmp/mapred/system (or
>>>whatever you configure it to) is to be owned by mapred:hadoop so that
>>>the JT can feel free to reconfigure it.
>>>
>>>- mapreduce.jobtracker.staging.root.dir (A HDFS dir that represents
>>>the parent directory for user's to write their per-user job stage
>>>files (JARs, etc.)) is set to "/user". The /user further contains each
>>>user's home directories, owned all by them. For example:
>>>
>>>drwxr-xr-x   - harsh    harsh 0 2012-09-27 15:51 /user/harsh
>>>
>>>All staging files from local user 'harsh' are hence written as the