Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> most efficient way to concatenate 3 tables into one?


Copy link to this message
-
Re: most efficient way to concatenate 3 tables into one?
Hi

Just follow the steps
1) Create an external table with location as /user/aggregatedTable
2) Move the contents of 3 monthly tables to this location
Hadoop fs -cp /user/monthlyTable1/* /user/aggregatedTable

Hadoop fs -cp /user/monthlyTable2/* /user/aggregatedTable

...

Replace the hsfs dirs in the above commands with those used in your tables.
Regards
Bejoy KS

Sent from handheld, please excuse typos.

-----Original Message-----
From: zuohua zhang <[EMAIL PROTECTED]>
Date: Wed, 12 Sep 2012 13:14:20
To: <[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>
Reply-To: [EMAIL PROTECTED]
Subject: Re: most efficient way to concatenate 3 tables into one?

Thanks Bejoy.
Yes, they have the same schema.
Can you explain further how to? I am new to hive.
On Wed, Sep 12, 2012 at 1:06 PM, Bejoy KS <[EMAIL PROTECTED]> wrote:

> **
> Hi
>
> If all the 3 tables have the same. Schema, Create an external table and
> move the data from all the 3 tables to this new table's location. Just a
> hdfs copy or move is not that expensive.
> Regards
> Bejoy KS
>
> Sent from handheld, please excuse typos.
> ------------------------------
> *From: * zuohua zhang <[EMAIL PROTECTED]>
> *Date: *Wed, 12 Sep 2012 13:02:42 -0700
> *To: *<[EMAIL PROTECTED]>
> *ReplyTo: * [EMAIL PROTECTED]
> *Subject: *most efficient way to concatenate 3 tables into one?
>
> I have 3 tables, each contains monthly data, I would like to make a table
> to concatenate all 3 months into 1 table.
> What is the most efficient way to do that?
> I am using insert into which takes a long time.
>
> Thanks!
>

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB