Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Can Hive 0.7 Rebuild partitions ?


Copy link to this message
-
Re: Can Hive 0.7 Rebuild partitions ?
I agree it's useful, especially for external tables, that may be loaded by
an external process that may 'forget' to issue a ADD PARTITION.
A 'sync partitions' feature to sync metadata with directories would be
really handy.

On Thu, May 19, 2011 at 3:23 PM, Igor Tatarinov <[EMAIL PROTECTED]> wrote:

> That's Amazon's extension to Hive and it's really handy.
>
>
> On Thu, May 19, 2011 at 2:01 PM, Tim Spence <[EMAIL PROTECTED]>wrote:
>
>> Is this functionality handled by ALTER TABLE [name] RECOVER PARTITIONS?
>> Take a look at this presentation for context:
>> http://www.slideshare.net/AmazonWebServices/aws-office-hours-amazon-elastic-mapreduce
>>
>> Best of luck,
>> Tim
>>
>>
>>
>>
>>
>> On Thu, May 19, 2011 at 2:25 AM, Jasper Knulst <[EMAIL PROTECTED]>wrote:
>>
>>> Hi,
>>>
>>> I have a partitioned external table on Hive 0.7. New subfolders are
>>> regularly added to the base table HDFS folder.
>>> I now have to perform this scan myself and let an external tool create
>>> new partitions by generating and firing ALTER TABLE ADD PARTITION commands.
>>>
>>> Is there an easier way to have hive scan the base table folder to see if
>>> there are any new partitions around? Something like REBUILD PARTITIONS
>>> perhaps??
>>>
>>>
>>>
>>>
>>> Couldn't find anything similar on the Hive/LanguageManual/DDL
>>>
>>> --
>>> Kind Regards
>>>
>>>
>>>
>>> Jasper
>>>
>>
>>
>
--
Roberto Congiu -Data Engineer - OpenX
20 E Del Mar blvd, Pasadena, CA
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB