Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive >> mail # user >> Create table like with partitions


+
Peter Marron 2013-06-10, 13:31
+
Nitin Pawar 2013-06-10, 13:38
+
Owen OMalley 2013-06-10, 14:06
+
Lefty Leverenz 2013-06-11, 07:03
+
Richa Sharma 2013-06-11, 08:23
+
Richa Sharma 2013-06-11, 12:34
Copy link to this message
-
Re: Create table like with partitions
For reference, any that puts the entire book online like this is likely
pirated.
On Tue, Jun 11, 2013 at 8:34 AM, Richa Sharma
<[EMAIL PROTECTED]>wrote:

> Hi all,
>
> Found a very useful guide online. Link -> http://it-ebooks.info/book/941/
>
> Richa
>
>
> On Tue, Jun 11, 2013 at 1:53 PM, Richa Sharma <[EMAIL PROTECTED]
> > wrote:
>
>> Thanks for sharing!
>>
>> I looked at these links .. Is there any documentation with more examples
>> with both static and dynamic partitions covered together.
>>
>> Richa
>>
>>
>> On Tue, Jun 11, 2013 at 12:33 PM, Lefty Leverenz <[EMAIL PROTECTED]>wrote:
>>
>>> Dynamic partitions are described in the Hive design docs here:
>>> https://cwiki.apache.org/confluence/display/Hive/DynamicPartitions.
>>>
>>> For the configuration parameters, though, you need to look in the
>>> language manual here:
>>> https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties (search
>>> for "dynamic" to find various parameters related to dynamic partitions).
>>>
>>> – Lefty
>>>
>>>
>>>
>>> On Mon, Jun 10, 2013 at 7:06 AM, Owen O'Malley <[EMAIL PROTECTED]>wrote:
>>>
>>>> You need to create the partitioned table and then copy the rows into it.
>>>>
>>>> create table foo_staging (int x, int y);
>>>>
>>>> create table foo(int x) partitioned by (int y) clustered by (x) into 16
>>>> buckets;
>>>>
>>>> set hive.exec.dynamic.partition=true;
>>>> set hive.exec.dynamic.partition.mode=nonstrict;
>>>> set hive.enforce.bucketing = true;
>>>>
>>>> insert overwrite table partition (y) select * from foo_staging;
>>>>
>>>>
>>>> On Mon, Jun 10, 2013 at 6:38 AM, Nitin Pawar <[EMAIL PROTECTED]>wrote:
>>>>
>>>>> If a table is not partitioned and then you want to partition the table
>>>>> on the data already written but data is not in partition format, that is
>>>>> not doable.
>>>>>
>>>>> Best approach would be, create a new table definition with the
>>>>> partition columns you want.
>>>>> turn on the dynamic partitioning system before you load data into new
>>>>> table
>>>>>
>>>>> set hive.exec.dynamic.partition=true;
>>>>> set hive.exec.dynamic.partition.mode=nonstrict;
>>>>>
>>>>> insert overwrite table partitioned(columns) select * from oldtable
>>>>>
>>>>>
>>>>> remove old table
>>>>>
>>>>> PS: wait for others to add more suggestions. I may be very well wrong
>>>>> in suggesting this
>>>>>
>>>>>
>>>>> On Mon, Jun 10, 2013 at 7:01 PM, Peter Marron <
>>>>> [EMAIL PROTECTED]> wrote:
>>>>>
>>>>>>  Hi,****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> Using hive 0.10.0 over hadoop 1.0.4****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> I have a (non-partitioned) table with loads of columns.****
>>>>>>
>>>>>> I would like to create a partitioned table with the same set of
>>>>>> columns.****
>>>>>>
>>>>>> So the approach that I have been taking is to use “CREATE TABLE copy
>>>>>> LIKE original;”****
>>>>>>
>>>>>> then I can use ALTER TABLE to change the location and the INPUTFORMAT
>>>>>> ****
>>>>>>
>>>>>> and the OUTPUTFORMAT and the SERDE and properties and pretty much****
>>>>>>
>>>>>> everything else. However I don’t seem to be able to make it
>>>>>> partitioned.****
>>>>>>
>>>>>> Sure I can add partitions if it’s already partitioned but I don’t seem
>>>>>> ****
>>>>>>
>>>>>> to be able to make it partitioned if it’s not already. I get errors
>>>>>> like this:****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> hive> ALTER TABLE customerShortValues ADD PARTITION (aid='1')
>>>>>> LOCATION 'E7/phase2/values/aid=1';****
>>>>>>
>>>>>> FAILED: Error in metadata: table is not partitioned but partition
>>>>>> spec exists: {aid=1}****
>>>>>>
>>>>>> FAILED: Execution Error, return code 1 from
>>>>>> org.apache.hadoop.hive.ql.exec.DDLTask****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> So, I guess that I could create the table I want by hand copying over
>>>>>> all the****
>>>>>>
>>>>>> column definitions. But is there an easier way?****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> Z****
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Nitin Pawar
+
Dean Wampler 2013-06-11, 18:04
+
Sanjay Subramanian 2013-06-11, 22:46
+
Lefty Leverenz 2013-06-12, 00:39
+
Sanjay Subramanian 2013-06-12, 02:11
+
Richa Sharma 2013-06-12, 06:20
+
Peter Marron 2013-06-12, 09:18
+
Richa Sharma 2013-06-10, 14:05
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB