Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # dev - Limiting the usage of counters


Copy link to this message
-
Re: Limiting the usage of counters
Praveen Sripati 2011-12-25, 11:25
Hi,

I found out the following in MRJobConfig.java in 0.23 and trunk and missing
in 0.22. The configuration parameters have been changed between 0.20.* to
0.23/trunk and so initially could not find it in the code.

  public static final String COUNTERS_MAX_KEY "mapreduce.job.counters.max";
  public static final int COUNTERS_MAX_DEFAULT = 120;

  public static final String COUNTER_GROUP_NAME_MAX_KEY "mapreduce.job.counters.group.name.max";
  public static final int COUNTER_GROUP_NAME_MAX_DEFAULT = 128;

  public static final String COUNTER_NAME_MAX_KEY "mapreduce.job.counters.counter.name.max";
  public static final int COUNTER_NAME_MAX_DEFAULT = 64;

  public static final String COUNTER_GROUPS_MAX_KEY "mapreduce.job.counters.groups.max";
  public static final int COUNTER_GROUPS_MAX_DEFAULT = 50;

Regards,
Praveen

On Sat, Dec 24, 2011 at 9:41 AM, Praveen Sripati
<[EMAIL PROTECTED]>wrote:

> Hi,
>
> I find the below code in 203, 205 and 1.0 and not in trunk and other
> releases. Is it not in trunk or done some other way? Also, noticed that
> some of the parameters are not configurable and hard-coded.
>
> /** limit on the size of the name of the group **
>   private static final int GROUP_NAME_LIMIT = 128;
>   /** limit on the size of the counter name **/
>   private static final int COUNTER_NAME_LIMIT = 64;
>
>   private static final JobConf conf = new JobConf();
>   /** limit on counters **/
>   public static int MAX_COUNTER_LIMIT >     conf.getInt("mapreduce.job.counters.limit", 120);
>
>   /** the max groups allowed **/
>   static final int MAX_GROUP_LIMIT = 50;
>
> Regards,
> Praveen
>