Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # dev >> Limiting the usage of counters


Copy link to this message
-
Re: Limiting the usage of counters
Hi,

I found out the following in MRJobConfig.java in 0.23 and trunk and missing
in 0.22. The configuration parameters have been changed between 0.20.* to
0.23/trunk and so initially could not find it in the code.

  public static final String COUNTERS_MAX_KEY "mapreduce.job.counters.max";
  public static final int COUNTERS_MAX_DEFAULT = 120;

  public static final String COUNTER_GROUP_NAME_MAX_KEY "mapreduce.job.counters.group.name.max";
  public static final int COUNTER_GROUP_NAME_MAX_DEFAULT = 128;

  public static final String COUNTER_NAME_MAX_KEY "mapreduce.job.counters.counter.name.max";
  public static final int COUNTER_NAME_MAX_DEFAULT = 64;

  public static final String COUNTER_GROUPS_MAX_KEY "mapreduce.job.counters.groups.max";
  public static final int COUNTER_GROUPS_MAX_DEFAULT = 50;

Regards,
Praveen

On Sat, Dec 24, 2011 at 9:41 AM, Praveen Sripati
<[EMAIL PROTECTED]>wrote:

> Hi,
>
> I find the below code in 203, 205 and 1.0 and not in trunk and other
> releases. Is it not in trunk or done some other way? Also, noticed that
> some of the parameters are not configurable and hard-coded.
>
> /** limit on the size of the name of the group **
>   private static final int GROUP_NAME_LIMIT = 128;
>   /** limit on the size of the counter name **/
>   private static final int COUNTER_NAME_LIMIT = 64;
>
>   private static final JobConf conf = new JobConf();
>   /** limit on counters **/
>   public static int MAX_COUNTER_LIMIT >     conf.getInt("mapreduce.job.counters.limit", 120);
>
>   /** the max groups allowed **/
>   static final int MAX_GROUP_LIMIT = 50;
>
> Regards,
> Praveen
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB