Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # user >> can I bulk import multiple mysql tables to one HBase/Hive table


Copy link to this message
-
Re: can I bulk import multiple mysql tables to one HBase/Hive table
Hi Shengjie,

You can try importing using --query and --boundary-query options.

Your query might look something like:
select * from A UNION select * from B UNION select * from C

And you boundary queries might look like:
select min(mintemp) from (select min(id) as mintemp from A UNION select
min(id) as mintemp from B UNION select min(id) as mintemp from C) t;

select max(maxtemp) from (select max(id) as maxtemp from A UNION select
max(id) as maxtemp from B UNION select max(id) as maxtemp from C) t;

If possible you can try creating a view and import that.

With Regards,
Raghav.

On Mon, Aug 5, 2013 at 1:02 AM, Shengjie Min <[EMAIL PROTECTED]> wrote:

> Hi guys,
>
> I am reasonable new to Sqoop, just have a quick question here:
>
> If I have multiple similar mysql tables, e.g:
>
> table A: "users", columns: user_name, user_id, user_address, etc etc
>
> table B: "customers" columns: customer_name, customer_id,
> customer_address, etc etc
>
> table C: "employee" columns: employee_name, employee_id, employe_address,
> etc etc
>
> Is it possible that using Sqoop to import the three tables into one HBase
> or Hive table? So After the import, I have one HBase table contains all the
> records in table A, B, C ?
>
> --
> All the best,
> Shengjie Min
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB