Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS, mail # user - 答复: Passing an object in mapper


Copy link to this message
-
答复: Passing an object in mapper
Francis.Hu 2013-08-15, 04:13
hi,

 

what I did when data need to be saved into DB :

 

//configure DB connection info.

DBConfiguration.configureDB(config,

                    HadoopJobConfigParser.getProperty(DB_DRIVER),

                    HadoopJobConfigParser.getProperty(DB_URL),

                    HadoopJobConfigParser.getProperty(DB_USER),

                    HadoopJobConfigParser.getProperty(DB_PWD));

 

//write a OutputKeyClass like below:

public class DisplayRequestRecord implements DBWritable,
WritableComparable<DisplayRequestRecord>{

    //TODO implement write() and readFiels() methods

}

 

//set the job's output

job.setOutputKeyClass(DisplayRequestRecord.class);

job.setOutputValueClass(DisplayRequestRecord.class);

DBOutputFormat.setOutput(job, tableName(),Fields());

 

//define reducer class like below:

public class DBDisplayRequestReducerNew extends Reducer<Text,
DoubleWritable, DisplayRequestRecord, DisplayRequestRecord>{

    public void reduce(Text key, Iterable<DoubleWritable> values,

            Context context) throws IOException, InterruptedException {

        DisplayRequestRecord dr = new DisplayRequestRecord();

        context.write(dr,dr);

    }

}

 

 

Hope it helps!

 

Thanks,

Francis.Hu

 

发件人: Sivaram RL [mailto:[EMAIL PROTECTED]]
发送时间: Thursday, August 15, 2013 11:26
收件人: [EMAIL PROTECTED]
主题: Re: Passing an object in mapper

 

Hi,

 

In the Configuration object in your driver class you can set the properties
as key value pair. This configuration object will be set in the Job Object.
The same properties can be accessed in the mapper/reducer using the Context
Object -> getConfiguration() -> get(propertyName).

 

Hope this helps.

 

Regards,

Sivaram R L

 

On Thu, Aug 15, 2013 at 8:39 AM, jamal sasha <[EMAIL PROTECTED]> wrote:

Hi,

   I am initializing an object in driver code.

For sake of argument let say I want to save data to some database..

say:

Connection con = new Connection(host, db);

Now, in reducer I want to do something like

con.write(key,value)

 

So, how do i pass this object from driver to mapper / reducer/?
Any clue? suggestions?

Thanks