Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # user - Hive Server Leaking File Descriptors?


Copy link to this message
-
Re: Hive Server Leaking File Descriptors?
Bennie Schut 2010-02-23, 09:02
fyi.. just tested this and I see no more leaks. Thanks.

Zheng Shao wrote:
> https://issues.apache.org/jira/browse/HIVE-1181 is committed.
> Thanks to Yonqqiang.
>
> We need to "-hiveconf hive.fileformat.check=false" when starting
> HiveServer to get rid of the extra connections.
>
> We still need to fix MAPREDUCE-1504 so that we can re-enable fileformat check.
>
> Zheng
>
> On Thu, Feb 18, 2010 at 11:47 PM, Bennie Schut <[EMAIL PROTECTED]> wrote:
>  
>> That's some great news. Thanks.
>>
>> Zheng Shao wrote:
>>    
>>> This is actually a bug in MAPREDUCE-1504, but we will try to find a
>>> workaround.
>>> https://issues.apache.org/jira/browse/HIVE-1181
>>>
>>> Given that release 0.5.0 is much wanted right now, I don't think we
>>> want to wait purely for 0.5.0 since the ultimate fix should come from
>>> Hadoop.
>>> We will definitely get HIVE-1181 for branch 0.5.
>>>
>>> Zheng
>>>
>>> ---------- Forwarded message ----------
>>> From: Andy Kent <[EMAIL PROTECTED]>
>>> Date: Thu, Feb 18, 2010 at 3:17 PM
>>> Subject: Re: Hive Server Leaking File Descriptors?
>>> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
>>>
>>>
>>>
>>> On 18 Feb 2010, at 20:29, "Zheng Shao" <[EMAIL PROTECTED]> wrote:
>>>
>>>
>>>      
>>>>> I've tried to look into it a bit more and it seems to happen on
>>>>> "load data
>>>>> inpath"
>>>>>
>>>>>          
>>> This is inline with what we have been seeing as we do around 200 load
>>> data statements per day and leak approx the same number of file
>>> descriptors.
>>>
>>> Is there any chance this fix will make it into the 0.5 release?
>>>
>>>
>>>
>>>
>>>      
>>    
>
>
>
>