|
|
-
Limiting the usage of counters
Praveen Sripati 2011-12-24, 04:11
Hi,
I find the below code in 203, 205 and 1.0 and not in trunk and other releases. Is it not in trunk or done some other way? Also, noticed that some of the parameters are not configurable and hard-coded.
/** limit on the size of the name of the group ** private static final int GROUP_NAME_LIMIT = 128; /** limit on the size of the counter name **/ private static final int COUNTER_NAME_LIMIT = 64;
private static final JobConf conf = new JobConf(); /** limit on counters **/ public static int MAX_COUNTER_LIMIT conf.getInt("mapreduce.job.counters.limit", 120);
/** the max groups allowed **/ static final int MAX_GROUP_LIMIT = 50;
Regards, Praveen
-
Re: Limiting the usage of counters
Praveen Sripati 2011-12-25, 11:25
Hi,
I found out the following in MRJobConfig.java in 0.23 and trunk and missing in 0.22. The configuration parameters have been changed between 0.20.* to 0.23/trunk and so initially could not find it in the code.
public static final String COUNTERS_MAX_KEY "mapreduce.job.counters.max"; public static final int COUNTERS_MAX_DEFAULT = 120;
public static final String COUNTER_GROUP_NAME_MAX_KEY "mapreduce.job.counters.group.name.max"; public static final int COUNTER_GROUP_NAME_MAX_DEFAULT = 128;
public static final String COUNTER_NAME_MAX_KEY "mapreduce.job.counters.counter.name.max"; public static final int COUNTER_NAME_MAX_DEFAULT = 64;
public static final String COUNTER_GROUPS_MAX_KEY "mapreduce.job.counters.groups.max"; public static final int COUNTER_GROUPS_MAX_DEFAULT = 50;
Regards, Praveen
On Sat, Dec 24, 2011 at 9:41 AM, Praveen Sripati <[EMAIL PROTECTED]>wrote:
> Hi, > > I find the below code in 203, 205 and 1.0 and not in trunk and other > releases. Is it not in trunk or done some other way? Also, noticed that > some of the parameters are not configurable and hard-coded. > > /** limit on the size of the name of the group ** > private static final int GROUP_NAME_LIMIT = 128; > /** limit on the size of the counter name **/ > private static final int COUNTER_NAME_LIMIT = 64; > > private static final JobConf conf = new JobConf(); > /** limit on counters **/ > public static int MAX_COUNTER_LIMIT > conf.getInt("mapreduce.job.counters.limit", 120); > > /** the max groups allowed **/ > static final int MAX_GROUP_LIMIT = 50; > > Regards, > Praveen >
|
|
All projects made searchable here are trademarks of the Apache Software Foundation.
Service operated by
Sematext