Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Problem with Hive JDBC server


Copy link to this message
-
Re: Re:RE: Problem with Hive JDBC server
  hmm .. there's a possibility that the execute and close for a given
statement are not handled by same thrift thread. Can you please verify the
testcase with a single worker thread on the server side ?
You can run the server using following :
hive --service hiveserver --maxWorkerThreads 1 --minWorkerThreads 1

thanks
Prasad

On Wed, Feb 6, 2013 at 3:44 AM, Gabor Makrai <[EMAIL PROTECTED]> wrote:

> Hi guys,
>
> Bad news for me. I checked out and compiled the Hive trunk and got the
> same problem.
> I attached to output of command lsof before and after my test program with
> 100 "SHOW TABLES" iterations. Is there any explanation why my JDBC server
> process doesn't release those files?
>
> Thanks,
> Gabor
>
>
> On Tue, Feb 5, 2013 at 6:20 AM, 王锋 <[EMAIL PROTECTED]> wrote:
>
>>
>>
>> I got it. pls see  https://issues.apache.org/jira/browse/THRIFT-1205
>>
>> I upgrade the thrift to libthrift-0.9.0.
>>
>> thanks
>>
>>
>>
>>
>> At 2013-02-05 13:06:05,"王锋" <[EMAIL PROTECTED]> wrote:
>>
>>
>> When I was using hiveserver ,the exception was thrown:
>>
>> 2060198 Hive history
>> file=/tmp/hdfs/hive_job_log_hdfs_201302010032_1918750748.txt
>> 2060199 Exception in thread "pool-1-thread-95"
>> java.lang.OutOfMemoryError: Java heap space
>> 2060200     at
>> org.apache.thrift.protocol.TBinaryProtocol.readStringBody(TBinaryProtocol.java:353)
>> 2060201     at
>> org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:215)
>> 2060202     at
>> org.apache.hadoop.hive.service.ThriftHive$Processor.process(ThriftHive.java:730)
>> 2060203     at
>> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:253)
>> 2060204     at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>> 2060205     at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>> 2060206     at java.lang.Thread.run(Thread.java:722)
>>
>> I using Hive-0.7.1-cdh3u1 with thrift-0.5.0.jar and
>> thrift-fb303-0.5.0.jar.
>> how can it be fixed? how about hive-0.7-1 using thrift -0.9.0?  thanks.
>>
>>
>>
>>
>> At 2013-02-04 19:19:16,"Bennie Schut" <[EMAIL PROTECTED]> wrote:
>>
>> Looking at the versions you might be hitting
>> https://issues.apache.org/jira/browse/HIVE-3481 which is fixed in 0.10***
>> *
>>
>> ** **
>>
>> On my dev machine the test runs with success :Running time: 298.952409914
>> ****
>>
>> This includes this patch so it’s worth looking at.****
>>
>> ** **
>>
>> *From:* Gabor Makrai [mailto:[EMAIL PROTECTED]]
>> *Sent:* Monday, February 04, 2013 11:58 AM
>> *To:* [EMAIL PROTECTED]
>> *Subject:* Re: Problem with Hive JDBC server****
>>
>> ** **
>>
>> Yes, of course! I attached the code!****
>>
>> ** **
>>
>> On Mon, Feb 4, 2013 at 11:57 AM, Gabor Makrai <[EMAIL PROTECTED]>
>> wrote:****
>>
>> Yes, of course! :) I attached the code!****
>>
>> ** **
>>
>> On Mon, Feb 4, 2013 at 11:53 AM, Bennie Schut <[EMAIL PROTECTED]> wrote:*
>> ***
>>
>> Since it’s small can you post the code?****
>>
>>  ****
>>
>> *From:* Gabor Makrai [mailto:[EMAIL PROTECTED]]
>> *Sent:* Monday, February 04, 2013 11:45 AM
>> *To:* [EMAIL PROTECTED]
>> *Subject:* Problem with Hive JDBC server****
>>
>>  ****
>>
>> Hi guys,****
>>
>>  ****
>>
>> I'm writing you because I experienced a very strange problem which
>> probably affects all Hive distribution.****
>>
>> I made a small "only main function" Java program where I'm only
>> connecting to my Hive JDBC, and getting the list of the database tables
>> (LIST TABLES) and closing the ResultSet, the Statement and the Connection
>> and doing this a 1000 times. The problem is that the running Hive JDBC
>> server does not release files and with time it will throw Exception
>> because, it will get "Too many open files" IOException from the JVM.****
>>
>>  ****
>>
>> I tested with Hive 0.9, 0.8.1, and the patched Hive 0.9 installed in
>> CDH4.1.1.****
>>
>>  ****
>>
>> If it is a know issue, than could you tell me the solution for it? If it
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB