Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> Writing MR-Job: Something like OracleReducer, JDBCReducer ...


+
Steinmaurer Thomas 2011-09-16, 05:25
+
Sonal Goyal 2011-09-16, 07:22
Copy link to this message
-
Re: Writing MR-Job: Something like OracleReducer, JDBCReducer ...
I think you need to get a little bit more information.
Reducers are expensive.
When Thomas says that he is aggregating data, what exactly does he mean?
When dealing w HBase, you really don't want to use a reducer.

You may want to run two map jobs and it could be that just dumping the output via jdbc makes the most sense.

We are starting to see a lot of questions where the OP isn't providing enough information so that the recommendation could be wrong...
Sent from a remote device. Please excuse any typos...

Mike Segel

On Sep 16, 2011, at 2:22 AM, Sonal Goyal <[EMAIL PROTECTED]> wrote:

> There is a DBOutputFormat class in the org.apache,hadoop.mapreduce.lib.db
> package, you could use that. Or you could write to the hdfs and then use
> something like HIHO[1] to export to the db. I have been working extensively
> in this area, you can write to me directly if you need any help.
>
> 1. https://github.com/sonalgoyal/hiho
>
> Best Regards,
> Sonal
> Crux: Reporting for HBase <https://github.com/sonalgoyal/crux>
> Nube Technologies <http://www.nubetech.co>
>
> <http://in.linkedin.com/in/sonalgoyal>
>
>
>
>
>
> On Fri, Sep 16, 2011 at 10:55 AM, Steinmaurer Thomas <
> [EMAIL PROTECTED]> wrote:
>
>> Hello,
>>
>>
>>
>> writing a MR-Job to process HBase data and store aggregated data in
>> Oracle. How would you do that in a MR-job?
>>
>>
>>
>> Currently, for test purposes we write the result into a HBase table
>> again by using a TableReducer. Is there something like a OracleReducer,
>> RelationalReducer, JDBCReducer or whatever? Or should one simply use
>> plan JDBC code in the reduce step?
>>
>>
>>
>> Thanks!
>>
>>
>>
>> Thomas
>>
>>
>>
>>
+
Sonal Goyal 2011-09-16, 16:06
+
Steinmaurer Thomas 2011-09-19, 05:35
+
Sonal Goyal 2011-09-16, 16:11
+
Michael Segel 2011-09-16, 17:05
+
Sonal Goyal 2011-09-16, 17:30
+
Michael Segel 2011-09-16, 18:43
+
Chris Tarnas 2011-09-16, 18:58
+
Doug Meil 2011-09-16, 19:41
+
Michael Segel 2011-09-16, 20:24
+
Steinmaurer Thomas 2011-09-19, 05:41
+
Doug Meil 2011-09-19, 13:35
+
Steinmaurer Thomas 2011-09-19, 13:44
+
Michael Segel 2011-09-16, 20:11
+
Chris Tarnas 2011-09-16, 21:54
+
Chris Tarnas 2011-09-16, 22:34
+
Sam Seigal 2011-09-17, 00:16
+
Doug Meil 2011-09-17, 00:22
+
Doug Meil 2011-09-17, 00:24
+
Sam Seigal 2011-09-17, 01:00
+
Doug Meil 2011-09-17, 01:14
+
Sam Seigal 2011-09-17, 01:39
+
Sam Seigal 2011-09-17, 01:44
+
Doug Meil 2011-09-17, 01:47
+
Michel Segel 2011-09-17, 13:12