Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> Partition classes, how to pass in background information


+
Jane Wayne 2012-03-14, 06:31
+
Chris White 2012-03-14, 10:06
Copy link to this message
-
Re: Partition classes, how to pass in background information
Thanks Chris! That worked!

On Wed, Mar 14, 2012 at 6:06 AM, Chris White <[EMAIL PROTECTED]>wrote:

> If your class implements the configurable interface, hadoop will call the
> setConf method after creating the instance. Look in the source code for
> ReflectionUtils.newInstance for more info
> On Mar 14, 2012 2:31 AM, "Jane Wayne" <[EMAIL PROTECTED]> wrote:
>
> > i am using the new org.apache.hadoop.mapreduce.Partitioner class.
> however,
> > i need to pass it some background information. how can i do this?
> >
> > in the old, org.apache.hadoop.mapred.Partitioner class (now deprecated),
> > this class extends JobConfigurable, and it seems the "hook" to pass in
> any
> > background data is with the JobConfigurable.configure(JobConf job)
> method.
> >
> > i thought that if i sub-classed org.apache.hadoop.mapreduce.Partitioner,
> i
> > could pass in the background information, however, in the
> > org.apache.hadoop.mapreduce.Job class, it only has a
> > setPartitionerClass(Class<? extends Partitioner>) method.
> >
> > all my development has been the new mapreduce package, and i would
> > definitely desire to stick with the new API/package. any help is
> > appreciated.
> >
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB