Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Can Hive 0.7 Rebuild partitions ?


Copy link to this message
-
Re: Can Hive 0.7 Rebuild partitions ?
I agree it's useful, especially for external tables, that may be loaded by
an external process that may 'forget' to issue a ADD PARTITION.
A 'sync partitions' feature to sync metadata with directories would be
really handy.

On Thu, May 19, 2011 at 3:23 PM, Igor Tatarinov <[EMAIL PROTECTED]> wrote:

> That's Amazon's extension to Hive and it's really handy.
>
>
> On Thu, May 19, 2011 at 2:01 PM, Tim Spence <[EMAIL PROTECTED]>wrote:
>
>> Is this functionality handled by ALTER TABLE [name] RECOVER PARTITIONS?
>> Take a look at this presentation for context:
>> http://www.slideshare.net/AmazonWebServices/aws-office-hours-amazon-elastic-mapreduce
>>
>> Best of luck,
>> Tim
>>
>>
>>
>>
>>
>> On Thu, May 19, 2011 at 2:25 AM, Jasper Knulst <[EMAIL PROTECTED]>wrote:
>>
>>> Hi,
>>>
>>> I have a partitioned external table on Hive 0.7. New subfolders are
>>> regularly added to the base table HDFS folder.
>>> I now have to perform this scan myself and let an external tool create
>>> new partitions by generating and firing ALTER TABLE ADD PARTITION commands.
>>>
>>> Is there an easier way to have hive scan the base table folder to see if
>>> there are any new partitions around? Something like REBUILD PARTITIONS
>>> perhaps??
>>>
>>>
>>>
>>>
>>> Couldn't find anything similar on the Hive/LanguageManual/DDL
>>>
>>> --
>>> Kind Regards
>>>
>>>
>>>
>>> Jasper
>>>
>>
>>
>
--
Roberto Congiu -Data Engineer - OpenX
20 E Del Mar blvd, Pasadena, CA