Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Re: 答复: Passing an object in mapper


Copy link to this message
-
Re: 答复: Passing an object in mapper
Thanks guys :)
On Wed, Aug 14, 2013 at 9:13 PM, Francis.Hu <[EMAIL PROTECTED]>wrote:

>  hi, ****
>
> ** **
>
> what I did when data need to be saved into DB :****
>
> ** **
>
> //configure DB connection info.****
>
> DBConfiguration.*configureDB*(config,****
>
>                     HadoopJobConfigParser.*getProperty*(*DB_DRIVER*),****
>
>                     HadoopJobConfigParser.*getProperty*(*DB_URL*),****
>
>                     HadoopJobConfigParser.*getProperty*(*DB_USER*),****
>
>                     HadoopJobConfigParser.*getProperty*(*DB_PWD*));****
>
> ** **
>
> //write a OutputKeyClass like below:****
>
> public class DisplayRequestRecord implements DBWritable,
> WritableComparable<DisplayRequestRecord>{****
>
>     //TODO implement write() and readFiels() methods****
>
> }****
>
> ** **
>
> //set the job's output****
>
> job.setOutputKeyClass(DisplayRequestRecord.class);****
>
> job.setOutputValueClass(DisplayRequestRecord.class);****
>
> *DBOutputFormat.setOutput(job, tableName(),Fields());*
>
> * *
>
> //define reducer class like below:****
>
> public class DBDisplayRequestReducerNew extends Reducer<Text,
> DoubleWritable, DisplayRequestRecord, DisplayRequestRecord>{****
>
>     *public* *void* reduce(Text key, Iterable<DoubleWritable> values,****
>
>             Context context) *throws* IOException, InterruptedException {*
> ***
>
>         DisplayRequestRecord dr = new DisplayRequestRecord();****
>
>         context.write(dr,dr);****
>
>     }****
>
> }****
>
> ** **
>
> ** **
>
> Hope it helps!****
>
> ** **
>
> Thanks,****
>
> Francis.Hu****
>
> ** **
>
> *发件人:* Sivaram RL [mailto:[EMAIL PROTECTED]]
> *发送时间:* Thursday, August 15, 2013 11:26
> *收件人:* [EMAIL PROTECTED]
> *主题:* Re: Passing an object in mapper****
>
> ** **
>
> Hi,****
>
> ** **
>
> In the Configuration object in your driver class you can set the
> properties as key value pair. This configuration object will be set in the
> Job Object. The same properties can be accessed in the mapper/reducer using
> the Context Object -> getConfiguration() -> get(propertyName).****
>
> ** **
>
> Hope this helps.****
>
> ** **
>
> Regards,****
>
> Sivaram R L****
>
> ** **
>
> On Thu, Aug 15, 2013 at 8:39 AM, jamal sasha <[EMAIL PROTECTED]>
> wrote:****
>
> Hi,****
>
>    I am initializing an object in driver code.****
>
> For sake of argument let say I want to save data to some database..****
>
> say:****
>
> Connection con = new Connection(host, db);****
>
> Now, in reducer I want to do something like****
>
> con.write(key,value) ****
>
> ** **
>
> So, how do i pass this object from driver to mapper / reducer/?
> Any clue? suggestions?****
>
> Thanks****
>
> ** **
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB