Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Closing long running hive query forcefully


Copy link to this message
-
Re: Closing long running hive query forcefully
That is a good way to do it. We do it with comment sometimes.

select /* myid bla bla*/ x,y,z

Edward

On Thu, Feb 7, 2013 at 11:12 AM, Gabor Makrai <[EMAIL PROTECTED]> wrote:
> Hi,
>
> We solved this problem in the following way (this is really not a simple
> solution):
> - start hive query in a different thread
> - we generated an unique id for the query and used the SET key=value;
> command (before the long query command) to give this unique id to the MR
> jobs related to the query
> - main thread waits for the completion or the cancellation sign (comes from
> the gui or anywhere else)
> - if cancellation sign comes, then we will kill the running query through
> the JobTracker
>   - list all jobs and search the job which contains the previously
> configured unique id
>   - kill that job
>   - that will cause exception on the query thread
>
> It's a little bit ugly, but thrift doesn't support async communication and
> so "statement.cancel()" function can't be supported/implemented.
>
> Gabor
>
>
>
> On Thu, Feb 7, 2013 at 2:21 PM, Nitin Pawar <[EMAIL PROTECTED]> wrote:
>>
>> hive waits till the hadoop job is completed. so unless you kill the job of
>> jdbc connection is dropped I don't see any other way to reduce the load on
>> application.
>> best when you think its  long enough, you will need to find a way to kill
>> the hadoop job. That will automatically release the resources in the
>> pipeline
>>
>>
>> On Thu, Feb 7, 2013 at 6:35 PM, Kugathasan Abimaran
>> <[EMAIL PROTECTED]> wrote:
>>>
>>> Hi All,
>>>
>>>
>>> Are there anyways to close the long running hive query through hive-jdbc?
>>> Since when ever Hive hangs, my application also hang, So I want to close the
>>> hive connection forcefully after a certain time.
>>>
>>>
>>>
>>> --
>>> Thanks,
>>> With Regards,
>>>
>>> Abimaran
>>
>>
>>
>>
>> --
>> Nitin Pawar
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB