-Re: Parameters that can be set per job
Hemanth Yamijala 2010-07-30, 04:28
> Is there a list of configuration parameters that can be set per job.
I'm almost certain there's no list that documents per-job settable
parameters that well. From 0.21 onwards, I think a convention adopted
is to name all job-related or task-related parameters to include 'job'
or 'map' or 'reduce' or 'task' in the name somewhere. These can be set
per job. The best option is to go over the documentation of any
parameters you are interested in, in the *-default.xml files.
> Specifically, can one set:
> - mapred.tasktracker.map.tasks.maximum
> - mapred.tasktracker.reduce.tasks.maximum
No, these are tasktracker specific parameters (as is indicated in the
name also). They cannot be set per job.
> - mapred.map.multithreadedrunner.threads
> - mapred.child.java.opts
> - mapred.task.timeout
These can be set (again, the naming convention is helpful)
> Also, I am trying to migrate from 0.18.3 to 0.20.2. In the new API JobConf
> is deprecated. How does one set the per job configuration parameters that
> were available in JobConf (e.g. mapred.map.max.attempts, etc.).
> I guess there must be documentation on this but I could not find it. I
> appreciate any advice you may have.