Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive >> mail # user >> Closing long running hive query forcefully


+
Kugathasan Abimaran 2013-02-07, 13:05
+
Nitin Pawar 2013-02-07, 13:21
Copy link to this message
-
Re: Closing long running hive query forcefully
Hi,

We solved this problem in the following way (this is really not a simple
solution):
- start hive query in a different thread
- we generated an unique id for the query and used the SET key=value;
command (before the long query command) to give this unique id to the MR
jobs related to the query
- main thread waits for the completion or the cancellation sign (comes from
the gui or anywhere else)
- if cancellation sign comes, then we will kill the running query through
the JobTracker
  - list all jobs and search the job which contains the previously
configured unique id
  - kill that job
  - that will cause exception on the query thread

It's a little bit ugly, but thrift doesn't support async communication and
so "statement.cancel()" function can't be supported/implemented.

Gabor

On Thu, Feb 7, 2013 at 2:21 PM, Nitin Pawar <[EMAIL PROTECTED]> wrote:

> hive waits till the hadoop job is completed. so unless you kill the job of
> jdbc connection is dropped I don't see any other way to reduce the load on
> application.
> best when you think its  long enough, you will need to find a way to kill
> the hadoop job. That will automatically release the resources in the
> pipeline
>
>
> On Thu, Feb 7, 2013 at 6:35 PM, Kugathasan Abimaran <
> [EMAIL PROTECTED]> wrote:
>
>> Hi All,
>>
>>
>> Are there anyways to close the long running hive query through hive-jdbc?
>> Since when ever Hive hangs, my application also hang, So I want to close
>> the hive connection forcefully after a certain time.
>>
>>
>>
>> --
>> Thanks,
>> With Regards,
>>
>> Abimaran
>>
>
>
>
> --
> Nitin Pawar
>
+
Edward Capriolo 2013-02-07, 16:31
+
Kugathasan Abimaran 2013-02-13, 06:10
+
wzc 2013-03-19, 04:17
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB