|
|
-
Are the values available in job.xml the actual values used for job
Bejoy Ks 2011-12-09, 06:44
Hi experts I have a query with the job.xml file in map reduce.I set some value in mapred-site.xml and *marked as final*, say mapred.num.reduce.tasks=15. When I submit my job I explicitly specified the number of reducers as -D mapred.num.reduce.tasks=4. Now as expected my my job should run with 15 reducers as I marked this value as final in my config file. Now what would be the value that would be available for mapred.num.reduce.tasks *in job.xml*, 15 or 4 ?
Thank you
Regards Bejoy.K.S
+
Bejoy Ks 2011-12-09, 06:44
-
Re: Are the values available in job.xml the actual values used for job
Bejoy Ks 2011-12-09, 06:48
Correction not mapred.num.reduce.tasks but mapred.reduce.tasks . :)
On Fri, Dec 9, 2011 at 12:14 PM, Bejoy Ks <[EMAIL PROTECTED]> wrote:
> Hi experts > I have a query with the job.xml file in map reduce.I set some > value in mapred-site.xml and *marked as final*, say > mapred.num.reduce.tasks=15. When I submit my job I explicitly specified the > number of reducers as -D mapred.num.reduce.tasks=4. Now as expected my my > job should run with 15 reducers as I marked this value as final in my > config file. Now what would be the value that would be available for > mapred.num.reduce.tasks *in job.xml*, 15 or 4 ? > > Thank you > > Regards > Bejoy.K.S >
+
Bejoy Ks 2011-12-09, 06:48
-
Re: Are the values available in job.xml the actual values used for job
Arun C Murthy 2011-12-09, 23:14
'final' is meant for admins to ensure certain values aren't overridable.
However, in the example you gave, you'll see 15 (since it's 'final').
Arun
On Dec 8, 2011, at 10:44 PM, Bejoy Ks wrote:
> Hi experts > I have a query with the job.xml file in map reduce.I set some value in mapred-site.xml and marked as final, say mapred.num.reduce.tasks=15. When I submit my job I explicitly specified the number of reducers as -D mapred.num.reduce.tasks=4. Now as expected my my job should run with 15 reducers as I marked this value as final in my config file. Now what would be the value that would be available for mapred.num.reduce.tasks in job.xml, 15 or 4 ? > > Thank you > > Regards > Bejoy.K.S
+
Arun C Murthy 2011-12-09, 23:14
-
Re: Are the values available in job.xml the actual values used for job
Bejoy Ks 2011-12-10, 17:43
Thanks Arun. To make my understanding a little more crisp. All the values for config parameters available in job.xml are the actual values used by a job for its execution. There won't be any further changes in these values at any point further down the lane of execution. (ie no changes in confing param values after the job.xml is generated). Is my understanding correct? I waned to make sure of this as in some clusters I'm working on I don't have acess to the config files *-site.xml . So I just want to make sure that job.xml is fully dependable and the right reference for the config values used by my job.
Thanks a lot!....
Regards Bejoy.KS
On Sat, Dec 10, 2011 at 4:44 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:
> 'final' is meant for admins to ensure certain values aren't overridable. > > However, in the example you gave, you'll see 15 (since it's 'final'). > > Arun > > On Dec 8, 2011, at 10:44 PM, Bejoy Ks wrote: > > Hi experts > I have a query with the job.xml file in map reduce.I set some > value in mapred-site.xml and *marked as final*, say > mapred.num.reduce.tasks=15. When I submit my job I explicitly specified the > number of reducers as -D mapred.num.reduce.tasks=4. Now as expected my my > job should run with 15 reducers as I marked this value as final in my > config file. Now what would be the value that would be available for > mapred.num.reduce.tasks *in job.xml*, 15 or 4 ? > > Thank you > > Regards > Bejoy.K.S > > >
+
Bejoy Ks 2011-12-10, 17:43
|
|