-Preferred way to submit a job?
David Rosenstrauch 2010-08-11, 22:13
What's the preferred way to submit a job these days?
org.apache.hadoop.mapreduce.Job.submit() ? Or
org.apache.hadoop.mapred.JobClient.runJob()? Or does it even matter?
(i.e., is there any difference between them?)
I've been trying to run a job using
org.apache.hadoop.mapreduce.Job.submit() (since I assumed that the
org.apache.hadoop.mapred.* classes were deprecated). However, I'm
seeing some weirdness (the "mapred.job.tracker" setting that I set on my
job's Configuration is getting ignored, and making the job get run
locally) and I was wondering if the way I was submitting my job might
have something to do with it.
On a related note, if there's actually no difference between the 2
methods, would anybody have any idea what could make the
"mapred.job.tracker" setting on a job Configuration get ignored? (I
currently have it set to "hdfs://<hadoop_job_tracker_host_name>:9001".)