Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - Writing MR-Job: Something like OracleReducer, JDBCReducer ...


Copy link to this message
-
Re: Writing MR-Job: Something like OracleReducer, JDBCReducer ...
Michel Segel 2011-09-16, 09:05
I think you need to get a little bit more information.
Reducers are expensive.
When Thomas says that he is aggregating data, what exactly does he mean?
When dealing w HBase, you really don't want to use a reducer.

You may want to run two map jobs and it could be that just dumping the output via jdbc makes the most sense.

We are starting to see a lot of questions where the OP isn't providing enough information so that the recommendation could be wrong...
Sent from a remote device. Please excuse any typos...

Mike Segel

On Sep 16, 2011, at 2:22 AM, Sonal Goyal <[EMAIL PROTECTED]> wrote:

> There is a DBOutputFormat class in the org.apache,hadoop.mapreduce.lib.db
> package, you could use that. Or you could write to the hdfs and then use
> something like HIHO[1] to export to the db. I have been working extensively
> in this area, you can write to me directly if you need any help.
>
> 1. https://github.com/sonalgoyal/hiho
>
> Best Regards,
> Sonal
> Crux: Reporting for HBase <https://github.com/sonalgoyal/crux>
> Nube Technologies <http://www.nubetech.co>
>
> <http://in.linkedin.com/in/sonalgoyal>
>
>
>
>
>
> On Fri, Sep 16, 2011 at 10:55 AM, Steinmaurer Thomas <
> [EMAIL PROTECTED]> wrote:
>
>> Hello,
>>
>>
>>
>> writing a MR-Job to process HBase data and store aggregated data in
>> Oracle. How would you do that in a MR-job?
>>
>>
>>
>> Currently, for test purposes we write the result into a HBase table
>> again by using a TableReducer. Is there something like a OracleReducer,
>> RelationalReducer, JDBCReducer or whatever? Or should one simply use
>> plan JDBC code in the reduce step?
>>
>>
>>
>> Thanks!
>>
>>
>>
>> Thomas
>>
>>
>>
>>