Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Partition classes, how to pass in background information


Copy link to this message
-
Partition classes, how to pass in background information
i am using the new org.apache.hadoop.mapreduce.Partitioner class. however,
i need to pass it some background information. how can i do this?

in the old, org.apache.hadoop.mapred.Partitioner class (now deprecated),
this class extends JobConfigurable, and it seems the "hook" to pass in any
background data is with the JobConfigurable.configure(JobConf job) method.

i thought that if i sub-classed org.apache.hadoop.mapreduce.Partitioner, i
could pass in the background information, however, in the
org.apache.hadoop.mapreduce.Job class, it only has a
setPartitionerClass(Class<? extends Partitioner>) method.

all my development has been the new mapreduce package, and i would
definitely desire to stick with the new API/package. any help is
appreciated.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB