Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Partition classes, how to pass in background information

Copy link to this message
Partition classes, how to pass in background information
i am using the new org.apache.hadoop.mapreduce.Partitioner class. however,
i need to pass it some background information. how can i do this?

in the old, org.apache.hadoop.mapred.Partitioner class (now deprecated),
this class extends JobConfigurable, and it seems the "hook" to pass in any
background data is with the JobConfigurable.configure(JobConf job) method.

i thought that if i sub-classed org.apache.hadoop.mapreduce.Partitioner, i
could pass in the background information, however, in the
org.apache.hadoop.mapreduce.Job class, it only has a
setPartitionerClass(Class<? extends Partitioner>) method.

all my development has been the new mapreduce package, and i would
definitely desire to stick with the new API/package. any help is