Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> External table with partitions


Copy link to this message
-
Re: External table with partitions
Sorry, I din understand your query on first look through.

Like Jagat said, you may need to go with a temp table for this.

Do a hadoop fs -cp ../../a.* <destn dir>

Create a external table with location as 'destn dir'.

CREATE EXERNAL TABLE <tmp tble name> LIKE <src table name> LOCATION '' ;

NB: I just gave the syntax from memory. please check the syntax in hive user guide.
Regards
Bejoy KS

Sent from remote device, Please excuse typos

-----Original Message-----
From: [EMAIL PROTECTED]
Date: Sun, 6 Jan 2013 14:39:45
To: <[EMAIL PROTECTED]>
Reply-To: [EMAIL PROTECTED]
Subject: Re: External table with partitions

Hi Oded

If you have created the directories manually that would come visible to the hive table only if the partitions/ sub dirs are added to the meta data using
'ALTER TABLE ... ADD PARTITION' .
Partitions are not retrieved implicitly into hive tabe even if you have a proper sub dir structure.

Similarly if you don't need a particular partition on your table permanently you can always delete them using the alter table command.

If you are intending to use a particular partition alone in your query no need to alter the partitions. Just append a where clause to the query that has scope only on the required partitions.

Hope this helps.

Regards
Bejoy KS

Sent from remote device, Please excuse typos

-----Original Message-----
From: Oded Poncz <[EMAIL PROTECTED]>
Date: Sun, 6 Jan 2013 16:07:26
To: <[EMAIL PROTECTED]>
Reply-To: [EMAIL PROTECTED]
Subject: External table with partitions

Is it possible to instruct hive to get only specific files from a
partitioned external table?
For example I have the following directory structure

data/dd=2012-12-31/a1.txt
data/dd=2012-12-31/a2.txt
data/dd=2012-12-31/a3.txt
data/dd=2012-12-31/a4.txt

data/dd=2012-12-31/b1.txt
data/dd=2012-12-31/b2.txt
data/dd=2012-12-31/b2.txt

Is it possible to add 2012-12-31 as a partition and tell hive to load only
the a* files to the table?
Thanks,