Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Writing MR-Job: Something like OracleReducer, JDBCReducer ...


Copy link to this message
-
Re: Writing MR-Job: Something like OracleReducer, JDBCReducer ...
I think you need to get a little bit more information.
Reducers are expensive.
When Thomas says that he is aggregating data, what exactly does he mean?
When dealing w HBase, you really don't want to use a reducer.

You may want to run two map jobs and it could be that just dumping the output via jdbc makes the most sense.

We are starting to see a lot of questions where the OP isn't providing enough information so that the recommendation could be wrong...
Sent from a remote device. Please excuse any typos...

Mike Segel

On Sep 16, 2011, at 2:22 AM, Sonal Goyal <[EMAIL PROTECTED]> wrote:

> There is a DBOutputFormat class in the org.apache,hadoop.mapreduce.lib.db
> package, you could use that. Or you could write to the hdfs and then use
> something like HIHO[1] to export to the db. I have been working extensively
> in this area, you can write to me directly if you need any help.
>
> 1. https://github.com/sonalgoyal/hiho
>
> Best Regards,
> Sonal
> Crux: Reporting for HBase <https://github.com/sonalgoyal/crux>
> Nube Technologies <http://www.nubetech.co>
>
> <http://in.linkedin.com/in/sonalgoyal>
>
>
>
>
>
> On Fri, Sep 16, 2011 at 10:55 AM, Steinmaurer Thomas <
> [EMAIL PROTECTED]> wrote:
>
>> Hello,
>>
>>
>>
>> writing a MR-Job to process HBase data and store aggregated data in
>> Oracle. How would you do that in a MR-job?
>>
>>
>>
>> Currently, for test purposes we write the result into a HBase table
>> again by using a TableReducer. Is there something like a OracleReducer,
>> RelationalReducer, JDBCReducer or whatever? Or should one simply use
>> plan JDBC code in the reduce step?
>>
>>
>>
>> Thanks!
>>
>>
>>
>> Thomas
>>
>>
>>
>>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB