Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> External table with partitions


Copy link to this message
-
Re: External table with partitions
Sorry, I din understand your query on first look through.

Like Jagat said, you may need to go with a temp table for this.

Do a hadoop fs -cp ../../a.* <destn dir>

Create a external table with location as 'destn dir'.

CREATE EXERNAL TABLE <tmp tble name> LIKE <src table name> LOCATION '' ;

NB: I just gave the syntax from memory. please check the syntax in hive user guide.
Regards
Bejoy KS

Sent from remote device, Please excuse typos

-----Original Message-----
From: [EMAIL PROTECTED]
Date: Sun, 6 Jan 2013 14:39:45
To: <[EMAIL PROTECTED]>
Reply-To: [EMAIL PROTECTED]
Subject: Re: External table with partitions

Hi Oded

If you have created the directories manually that would come visible to the hive table only if the partitions/ sub dirs are added to the meta data using
'ALTER TABLE ... ADD PARTITION' .
Partitions are not retrieved implicitly into hive tabe even if you have a proper sub dir structure.

Similarly if you don't need a particular partition on your table permanently you can always delete them using the alter table command.

If you are intending to use a particular partition alone in your query no need to alter the partitions. Just append a where clause to the query that has scope only on the required partitions.

Hope this helps.

Regards
Bejoy KS

Sent from remote device, Please excuse typos

-----Original Message-----
From: Oded Poncz <[EMAIL PROTECTED]>
Date: Sun, 6 Jan 2013 16:07:26
To: <[EMAIL PROTECTED]>
Reply-To: [EMAIL PROTECTED]
Subject: External table with partitions

Is it possible to instruct hive to get only specific files from a
partitioned external table?
For example I have the following directory structure

data/dd=2012-12-31/a1.txt
data/dd=2012-12-31/a2.txt
data/dd=2012-12-31/a3.txt
data/dd=2012-12-31/a4.txt

data/dd=2012-12-31/b1.txt
data/dd=2012-12-31/b2.txt
data/dd=2012-12-31/b2.txt

Is it possible to add 2012-12-31 as a partition and tell hive to load only
the a* files to the table?
Thanks,

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB