Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> inflight jobs and updating a partition location


Copy link to this message
-
Re: inflight jobs and updating a partition location
Hi Tim,
I guess there are several ways to do it and your method seems to be one of
them.   I have a need for the same thing and i create a view instead. It
points to the date that is the latest partition.

eg. create view foo_latest_vw as select * from foo_table where date=<the
latest date> #and date is the partition key.

I'm sure there are other ways as well. :)

On Wed, Oct 2, 2013 at 12:45 PM, Timothy Potter <[EMAIL PROTECTED]>wrote:

> btw ... this appears to work in my env - hive 0.9.0 (cdh 4.1.1) ... let me
> know if there's any drawbacks to this approach.
>
> Thanks.
> Tim
>
>
> On Wed, Oct 2, 2013 at 1:12 PM, Timothy Potter <[EMAIL PROTECTED]>wrote:
>
>> Hi,
>>
>> I'd like to implement a "latest" partition concept for one of my tables
>> and believe I can simply update the location using alter table X partition
>> (date='latest') set location 'foo';
>>
>> This assumes two partitions can point at the same location?
>>
>> My other question is what happens to current running jobs against the
>> partition I'm updating. In my case the jobs are Pig scripts using
>> HCatLoader. I believe the HCatLoader reads input paths during job setup, so
>> changing the location of my "latest" partition is safe, but wanted to
>> verify.
>>
>> Thanks.
>> Tim
>>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB