Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Parameters that can be set per job


Copy link to this message
-
Re: Parameters that can be set per job
Hi,

> Is there a list of configuration parameters that can be set per job.

I'm almost certain there's no list that documents per-job settable
parameters that well. From 0.21 onwards, I think a convention adopted
is to name all job-related or task-related parameters to include 'job'
or 'map' or 'reduce' or 'task' in the name somewhere. These can be set
per job. The best option is to go over the documentation of any
parameters you are interested in, in the *-default.xml files.

> Specifically, can one set:
>
> - mapred.tasktracker.map.tasks.maximum
> - mapred.tasktracker.reduce.tasks.maximum

No, these are tasktracker specific parameters (as is indicated in the
name also). They cannot be set per job.

> - mapred.map.multithreadedrunner.threads
> - mapred.child.java.opts
> - mapred.task.timeout

These can be set (again, the naming convention is helpful)

> Also, I am trying to migrate from 0.18.3 to 0.20.2. In the new API JobConf
> is deprecated. How does one set the per job configuration parameters that
> were available in JobConf (e.g. mapred.map.max.attempts, etc.).
>
> I guess there must be documentation on this but I could not find it. I
> appreciate any advice you may have.
>
> Cheers,
> Dev
>