Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> inflight jobs and updating a partition location


Copy link to this message
-
Re: inflight jobs and updating a partition location
I think there was an issue with reading views from Pig using HCatLoader
(0.4.x). Views definitely seem cleaner though.
On Wed, Oct 2, 2013 at 2:06 PM, Stephen Sprague <[EMAIL PROTECTED]> wrote:

> Hi Tim,
> I guess there are several ways to do it and your method seems to be one of
> them.   I have a need for the same thing and i create a view instead. It
> points to the date that is the latest partition.
>
> eg. create view foo_latest_vw as select * from foo_table where date=<the
> latest date> #and date is the partition key.
>
> I'm sure there are other ways as well. :)
>
>
>
>
>
> On Wed, Oct 2, 2013 at 12:45 PM, Timothy Potter <[EMAIL PROTECTED]>wrote:
>
>> btw ... this appears to work in my env - hive 0.9.0 (cdh 4.1.1) ... let
>> me know if there's any drawbacks to this approach.
>>
>> Thanks.
>> Tim
>>
>>
>> On Wed, Oct 2, 2013 at 1:12 PM, Timothy Potter <[EMAIL PROTECTED]>wrote:
>>
>>> Hi,
>>>
>>> I'd like to implement a "latest" partition concept for one of my tables
>>> and believe I can simply update the location using alter table X partition
>>> (date='latest') set location 'foo';
>>>
>>> This assumes two partitions can point at the same location?
>>>
>>> My other question is what happens to current running jobs against the
>>> partition I'm updating. In my case the jobs are Pig scripts using
>>> HCatLoader. I believe the HCatLoader reads input paths during job setup, so
>>> changing the location of my "latest" partition is safe, but wanted to
>>> verify.
>>>
>>> Thanks.
>>> Tim
>>>
>>
>>
>