Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # dev - ConfigKeys wrappers for MapReduce source code base


Copy link to this message
-
Re: ConfigKeys wrappers for MapReduce source code base
Sandy Ryza 2013-04-22, 06:05
Hi Jay,

In trunk, these keys are in MRJobConfig (extended by JobContext) and
MRConfig.  I'm not sure what the state of things are in branch-1.  Agreed
that it would be nice for the naming convention to be consistent with HDFS,
but personally I like *Config more than *ConfigKeys better because the
class holds default values as well.

-Sandy
On Sun, Apr 21, 2013 at 12:29 PM, Jay Vyas <[EMAIL PROTECTED]> wrote:

> Hi guys:
>
> A breif check wiht "find ./ -name *ConfigKeys*" doesn't seem to indicate
> that there is a "MapRedConfigKeys" class... Should there be one to help get
> rid of magic numbers and unify the namespace?  This seems to be the goal of
> the DFSConfigKeys class in the HDFS source tree.
>
> ... Or are there differences in the way configuration values are handled in
> mapred versus hdfs code bases ?
>
> For example :
>
> "job.getInt(JobContext.IO_SORT_FACTOR, 100)" in the ReduceTask class would
> more typically be implemented (if in hdfs) using the DFSConfigKeys static
> class which stores defaults and configuration parameter names.
>
> Just curious wether there is any goal to take the Mapred configuration
> values and unify their namespace in mapred/common in the same way that
> seems to have been done in hdfs using the DFSConfigKeys class.
>
> --
> Jay Vyas
> http://jayunit100.blogspot.com
>