Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop, mail # user - can I bulk import multiple mysql tables to one HBase/Hive table


Copy link to this message
-
Re: can I bulk import multiple mysql tables to one HBase/Hive table
Raghav Gautam 2013-08-05, 21:06
Hi Shengjie,

You can try importing using --query and --boundary-query options.

Your query might look something like:
select * from A UNION select * from B UNION select * from C

And you boundary queries might look like:
select min(mintemp) from (select min(id) as mintemp from A UNION select
min(id) as mintemp from B UNION select min(id) as mintemp from C) t;

select max(maxtemp) from (select max(id) as maxtemp from A UNION select
max(id) as maxtemp from B UNION select max(id) as maxtemp from C) t;

If possible you can try creating a view and import that.

With Regards,
Raghav.

On Mon, Aug 5, 2013 at 1:02 AM, Shengjie Min <[EMAIL PROTECTED]> wrote:

> Hi guys,
>
> I am reasonable new to Sqoop, just have a quick question here:
>
> If I have multiple similar mysql tables, e.g:
>
> table A: "users", columns: user_name, user_id, user_address, etc etc
>
> table B: "customers" columns: customer_name, customer_id,
> customer_address, etc etc
>
> table C: "employee" columns: employee_name, employee_id, employe_address,
> etc etc
>
> Is it possible that using Sqoop to import the three tables into one HBase
> or Hive table? So After the import, I have one HBase table contains all the
> records in table A, B, C ?
>
> --
> All the best,
> Shengjie Min
>