Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> Bulk Loading from Oracle to Hbase


+
Mehmet Simsek 2012-12-13, 15:46
+
Mohammad Tariq 2012-12-13, 15:48
+
Mehmet Simsek 2012-12-13, 15:57
+
Manoj Babu 2012-12-13, 16:44
+
Amandeep Khurana 2012-12-13, 16:48
Copy link to this message
-
Re: Bulk Loading from Oracle to Hbase
Hi Mehmet,

My suggestion would be:

1. Use Sqoop to copy the data from Oracle into HDFS.
2. Use HBase Bulk Loader to load the data into HBase Table.

Sqoop implementation of loading data directly into HBase uses HBase Api
which is not as efficient as HBase Bulk Loader. Also, features provided by
Sqoop for loading data into HBase are limited.

HTH,
Anil Gupta
On Thu, Dec 13, 2012 at 8:48 AM, Amandeep Khurana <[EMAIL PROTECTED]> wrote:

> Mehmet
>
> What's the problem you are getting while running the Sqoop job? Can
> you give details?
>
> -Amandeep
>
> On Thu, Dec 13, 2012 at 8:44 AM, Manoj Babu <[EMAIL PROTECTED]> wrote:
> > Mehmet,
> >
> > You can try to write a MapReduce using DBInputFormat and insert into
> HBase.
> >
> http://hadoop.apache.org/docs/r0.20.0/api/org/apache/hadoop/mapred/lib/db/DBInputFormat.html
> >
> >
> > Cheers!
> > Manoj.
> >
> >
> >
> > On Thu, Dec 13, 2012 at 9:27 PM, Mehmet Simsek <[EMAIL PROTECTED]
> >wrote:
> >
> >> Thanks to answer Mohammad.I tried sqoop but there are biliions rows.
> >> Select query processing too long.namenode does not get respond from
> daanode
> >> in 600sec. And throws exception. I can increase this duration but select
> >> query running too long may be 6 hours. Is there a another good
> solution? Or
> >> another technique in sqoop?
> >>
> >> M.S
> >>
> >> On 13 Ara 2012, at 17:48, Mohammad Tariq <[EMAIL PROTECTED]> wrote:
> >>
> >> > Hello Mehmet,
> >> >
> >> >    Have a look at Acache Sqoop <http://sqoop.apache.org/>
> >> >
> >> > Regards,
> >> >    Mohammad Tariq
> >> >
> >> >
> >> >
> >> > On Thu, Dec 13, 2012 at 9:16 PM, Mehmet Simsek <
> [EMAIL PROTECTED]
> >> >wrote:
> >> >
> >> >> Hi,I want to load 5 billion rows from oracle table to hbase table.
> Which
> >> >> technique is the best for this bulk loading?
> >> >>
> >> >> Thanks
> >> >>
> >> >> M.S
> >> >>
> >> >>
> >>
>

--
Thanks & Regards,
Anil Gupta
+
Azuryy Yu 2012-12-14, 03:11
+
Mehmet Simsek 2012-12-14, 03:36
+
Azuryy Yu 2012-12-14, 05:50
+
Manoj Babu 2012-12-14, 03:35