Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> HiveServer2


Copy link to this message
-
Re: HiveServer2
Yeah, we had issues with zk locking too....but this workaround works pretty
well and our ops won't get paged at 4am :)
On Mon, Nov 18, 2013 at 2:16 PM, Edward Capriolo <[EMAIL PROTECTED]>wrote:

> Yikes: I personally found that the most problematic thing was hiveserver +
> zk locking, if you do not need that turn it off. Other then that we just
> wrote a good nagios check..it runs a query (one that does not invoke a map
> reduce job). That seems to spot the problems quickly and allow our ops to
> restart the bad instance.
>
>
> On Mon, Nov 18, 2013 at 5:11 PM, Roberto Congiu <[EMAIL PROTECTED]>wrote:
>
>> We've also had issues with both hiveserver1 and 2 crashing because of
>> heap exhaustion, but instead of restarting it periodically we took a
>> different approach, that is, abstracting the part of the interface we
>> needed, and implemented an adapter that implements the same method as
>> thrift, but forking a shell, sending commands to it, and parsing the
>> results.
>> It is slow, but it's fast enough for our hourly process that loads data
>> in hive EXTERNAL tables for which we need extra reliability.
>>
>> R.
>>
>>
>> On Mon, Nov 18, 2013 at 1:24 PM, Edward Capriolo <[EMAIL PROTECTED]>wrote:
>>
>>> Thanks for pointing out any issue. HiveServer1 is significantly less
>>> robust. We have run HS1 behind a load balancer/proxy and rotated/restarted
>>> "angry" instances.
>>>
>>>
>>> On Mon, Nov 18, 2013 at 3:59 PM, Stephen Sprague <[EMAIL PROTECTED]>wrote:
>>>
>>>> A word of warning for users of HiveServer2 - version 0.11 at least.
>>>> This puppy has the ability crash and/or hang your server with a memory leak.
>>>>
>>>> Apparently its not new since googling shows this discussed before and i
>>>> see reference to a workaround here:
>>>>
>>>> https://cwiki.apache.org/confluence/display/Hive/Setting+up+HiveServer2
>>>>
>>>> Anyhoo. Consider this a Public Service Announcement. Take heed.
>>>>
>>>> Regards,
>>>> Stephen.
>>>>
>>>>
>>>>
>>>>
>>>
>>
>>
>> --
>> ----------------------------------------------------------
>> Good judgement comes with experience.
>> Experience comes with bad judgement.
>> ----------------------------------------------------------
>> Roberto Congiu - Data Engineer - OpenX
>> tel: +1 626 466 1141
>>
>
>
--
----------------------------------------------------------
Good judgement comes with experience.
Experience comes with bad judgement.
----------------------------------------------------------
Roberto Congiu - Data Engineer - OpenX
tel: +1 626 466 1141