Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # user - inflight jobs and updating a partition location


Copy link to this message
-
Re: inflight jobs and updating a partition location
Stephen Sprague 2013-10-02, 20:06
Hi Tim,
I guess there are several ways to do it and your method seems to be one of
them.   I have a need for the same thing and i create a view instead. It
points to the date that is the latest partition.

eg. create view foo_latest_vw as select * from foo_table where date=<the
latest date> #and date is the partition key.

I'm sure there are other ways as well. :)

On Wed, Oct 2, 2013 at 12:45 PM, Timothy Potter <[EMAIL PROTECTED]>wrote:

> btw ... this appears to work in my env - hive 0.9.0 (cdh 4.1.1) ... let me
> know if there's any drawbacks to this approach.
>
> Thanks.
> Tim
>
>
> On Wed, Oct 2, 2013 at 1:12 PM, Timothy Potter <[EMAIL PROTECTED]>wrote:
>
>> Hi,
>>
>> I'd like to implement a "latest" partition concept for one of my tables
>> and believe I can simply update the location using alter table X partition
>> (date='latest') set location 'foo';
>>
>> This assumes two partitions can point at the same location?
>>
>> My other question is what happens to current running jobs against the
>> partition I'm updating. In my case the jobs are Pig scripts using
>> HCatLoader. I believe the HCatLoader reads input paths during job setup, so
>> changing the location of my "latest" partition is safe, but wanted to
>> verify.
>>
>> Thanks.
>> Tim
>>
>
>