Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> most efficient way to concatenate 3 tables into one?


Copy link to this message
-
Re: most efficient way to concatenate 3 tables into one?
Hi

Just follow the steps
1) Create an external table with location as /user/aggregatedTable
2) Move the contents of 3 monthly tables to this location
Hadoop fs -cp /user/monthlyTable1/* /user/aggregatedTable

Hadoop fs -cp /user/monthlyTable2/* /user/aggregatedTable

...

Replace the hsfs dirs in the above commands with those used in your tables.
Regards
Bejoy KS

Sent from handheld, please excuse typos.

-----Original Message-----
From: zuohua zhang <[EMAIL PROTECTED]>
Date: Wed, 12 Sep 2012 13:14:20
To: <[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>
Reply-To: [EMAIL PROTECTED]
Subject: Re: most efficient way to concatenate 3 tables into one?

Thanks Bejoy.
Yes, they have the same schema.
Can you explain further how to? I am new to hive.
On Wed, Sep 12, 2012 at 1:06 PM, Bejoy KS <[EMAIL PROTECTED]> wrote:

> **
> Hi
>
> If all the 3 tables have the same. Schema, Create an external table and
> move the data from all the 3 tables to this new table's location. Just a
> hdfs copy or move is not that expensive.
> Regards
> Bejoy KS
>
> Sent from handheld, please excuse typos.
> ------------------------------
> *From: * zuohua zhang <[EMAIL PROTECTED]>
> *Date: *Wed, 12 Sep 2012 13:02:42 -0700
> *To: *<[EMAIL PROTECTED]>
> *ReplyTo: * [EMAIL PROTECTED]
> *Subject: *most efficient way to concatenate 3 tables into one?
>
> I have 3 tables, each contains monthly data, I would like to make a table
> to concatenate all 3 months into 1 table.
> What is the most efficient way to do that?
> I am using insert into which takes a long time.
>
> Thanks!
>