Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Bulk Loading from Oracle to Hbase


Copy link to this message
-
Re: Bulk Loading from Oracle to Hbase
Then, I suggest:
1) Increase your session per user in RAC equal or greater than your map
tasks
2) I suppose you want 100 map tasks, then write a little java code to
generate 100 files, each file has only one line, you can write any in this
line.
3) Define a STEP for all maps. then each map task select STEP records from
DB.
4) Put all your files to the HDFS
5) Create Map(no reduce here),  execute "select * from Oracle where rownum
>= lineno and rownum < maptask_id * STEP" in each map task.
On Fri, Dec 14, 2012 at 11:36 AM, Mehmet Simsek <[EMAIL PROTECTED]>wrote:

> Hello Azuryy,
>
> Oracle in RAC.because of too long running sql statement datanode cannot
> respond to namenode about statistics.because of that exception is thrown by
> namenode like as "cannot respond from data node in 600sec..." In sqoop tool.
>
> Is there an another solution about this problem?
>
> M.S.
>
> On 14 Ara 2012, at 05:11, Azuryy Yu <[EMAIL PROTECTED]> wrote:
>
> > Hello Mehmet,
> >
> > what about your oracle? a single node or RAC ?
> >
> >
> > On Thu, Dec 13, 2012 at 11:46 PM, Mehmet Simsek <
> [EMAIL PROTECTED]>wrote:
> >
> >> Hi,I want to load 5 billion rows from oracle table to hbase table. Which
> >> technique is the best for this bulk loading?
> >>
> >> Thanks
> >>
> >> M.S
> >>
> >>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB