Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> create external table on existing hive partitioned table ?


Copy link to this message
-
Re: create external table on existing hive partitioned table ?
Hi Sai,
You might want to look into Alter table statements to see the type of the column can be modified. You may need to upgrade Hive if that functionality is not available in your version.

Mark

----- Original Message -----
From: "Sai Naga" <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]
Sent: Tuesday, June 19, 2012 6:16:31 PM
Subject: Re: create external table on existing hive partitioned table ?

Thanks Mark,
The reason to create the 2nd table is One of the column is defined as string in the first table, I wanted to read the string into Map data type.
i.e
Existing table.
{"UY": 2, "BR": 1}
{"LV": 1, "BR": 1}

To
Country Map<Sting, int >

Thanks
Gopi
On Tue, Jun 19, 2012 at 1:37 PM, Mark Grover < [EMAIL PROTECTED] > wrote:
Sai,
Maybe I don't understand your question properly but creating an external table on a partitioned table is no different than create an external table on a non-partitioned one.

Your syntax looks right. After table creation, you would have to add all existing partitions of the table so that the new table's metadata is properly populated.
You can do that using a command like "ALTER TABLE page_access_m RECOVER PARTITIONS" on Amazon EMR Hive or using msck on Apache Hive (more details at: https://issues.apache.org/jira/browse/HIVE-874 )

Does that answer your question?

On a related note, having two tables over the same data could be troublesome. For example, if one query adds a partition to one table, you would have to remember to recover partitions on the on the other table before you access your data through it. If the schema for both the tables is the same (except the fact that one is managed while other is external), any particular reason you'd like to create a new table?

Mark

----- Original Message -----
From: "Sai Naga" < [EMAIL PROTECTED] >
To: [EMAIL PROTECTED]
Sent: Tuesday, June 19, 2012 4:19:25 PM
Subject: create external table on existing hive partitioned table ?
Is it possible, to create external table on a existing hive table which is partitioned.
I have a existing hive table which is partitioned by dt and group like below
desc page_access;
page string
country string
dt string ( Partitioned column )
group string ( Partitioned column )
dfs -tail file /user/hive/warehouse/page_access/dt=2012-06-01/group=a/000003_0
PXXXXX1 {"UY": 2, "BR": 1}
PXXXXX2 {"LV": 1, "BR": 1}
Is it possible to define a external table something like..
create external table page_access_m
(
page string
country map(<string>,<int>)
)PARTITIONED BY(dt STRING, group STRING)
location /user/hive/warehouse/page_access;
currently I'm using hive.0.6
Thanks
Sai
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB