Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - Re: 答复: Passing an object in mapper


Copy link to this message
-
Re: 答复: Passing an object in mapper
jamal sasha 2013-08-15, 04:41
Thanks guys :)
On Wed, Aug 14, 2013 at 9:13 PM, Francis.Hu <[EMAIL PROTECTED]>wrote:

>  hi, ****
>
> ** **
>
> what I did when data need to be saved into DB :****
>
> ** **
>
> //configure DB connection info.****
>
> DBConfiguration.*configureDB*(config,****
>
>                     HadoopJobConfigParser.*getProperty*(*DB_DRIVER*),****
>
>                     HadoopJobConfigParser.*getProperty*(*DB_URL*),****
>
>                     HadoopJobConfigParser.*getProperty*(*DB_USER*),****
>
>                     HadoopJobConfigParser.*getProperty*(*DB_PWD*));****
>
> ** **
>
> //write a OutputKeyClass like below:****
>
> public class DisplayRequestRecord implements DBWritable,
> WritableComparable<DisplayRequestRecord>{****
>
>     //TODO implement write() and readFiels() methods****
>
> }****
>
> ** **
>
> //set the job's output****
>
> job.setOutputKeyClass(DisplayRequestRecord.class);****
>
> job.setOutputValueClass(DisplayRequestRecord.class);****
>
> *DBOutputFormat.setOutput(job, tableName(),Fields());*
>
> * *
>
> //define reducer class like below:****
>
> public class DBDisplayRequestReducerNew extends Reducer<Text,
> DoubleWritable, DisplayRequestRecord, DisplayRequestRecord>{****
>
>     *public* *void* reduce(Text key, Iterable<DoubleWritable> values,****
>
>             Context context) *throws* IOException, InterruptedException {*
> ***
>
>         DisplayRequestRecord dr = new DisplayRequestRecord();****
>
>         context.write(dr,dr);****
>
>     }****
>
> }****
>
> ** **
>
> ** **
>
> Hope it helps!****
>
> ** **
>
> Thanks,****
>
> Francis.Hu****
>
> ** **
>
> *发件人:* Sivaram RL [mailto:[EMAIL PROTECTED]]
> *发送时间:* Thursday, August 15, 2013 11:26
> *收件人:* [EMAIL PROTECTED]
> *主题:* Re: Passing an object in mapper****
>
> ** **
>
> Hi,****
>
> ** **
>
> In the Configuration object in your driver class you can set the
> properties as key value pair. This configuration object will be set in the
> Job Object. The same properties can be accessed in the mapper/reducer using
> the Context Object -> getConfiguration() -> get(propertyName).****
>
> ** **
>
> Hope this helps.****
>
> ** **
>
> Regards,****
>
> Sivaram R L****
>
> ** **
>
> On Thu, Aug 15, 2013 at 8:39 AM, jamal sasha <[EMAIL PROTECTED]>
> wrote:****
>
> Hi,****
>
>    I am initializing an object in driver code.****
>
> For sake of argument let say I want to save data to some database..****
>
> say:****
>
> Connection con = new Connection(host, db);****
>
> Now, in reducer I want to do something like****
>
> con.write(key,value) ****
>
> ** **
>
> So, how do i pass this object from driver to mapper / reducer/?
> Any clue? suggestions?****
>
> Thanks****
>
> ** **
>