Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> HiveServer2


Copy link to this message
-
Re: HiveServer2
Yeah, we had issues with zk locking too....but this workaround works pretty
well and our ops won't get paged at 4am :)
On Mon, Nov 18, 2013 at 2:16 PM, Edward Capriolo <[EMAIL PROTECTED]>wrote:

> Yikes: I personally found that the most problematic thing was hiveserver +
> zk locking, if you do not need that turn it off. Other then that we just
> wrote a good nagios check..it runs a query (one that does not invoke a map
> reduce job). That seems to spot the problems quickly and allow our ops to
> restart the bad instance.
>
>
> On Mon, Nov 18, 2013 at 5:11 PM, Roberto Congiu <[EMAIL PROTECTED]>wrote:
>
>> We've also had issues with both hiveserver1 and 2 crashing because of
>> heap exhaustion, but instead of restarting it periodically we took a
>> different approach, that is, abstracting the part of the interface we
>> needed, and implemented an adapter that implements the same method as
>> thrift, but forking a shell, sending commands to it, and parsing the
>> results.
>> It is slow, but it's fast enough for our hourly process that loads data
>> in hive EXTERNAL tables for which we need extra reliability.
>>
>> R.
>>
>>
>> On Mon, Nov 18, 2013 at 1:24 PM, Edward Capriolo <[EMAIL PROTECTED]>wrote:
>>
>>> Thanks for pointing out any issue. HiveServer1 is significantly less
>>> robust. We have run HS1 behind a load balancer/proxy and rotated/restarted
>>> "angry" instances.
>>>
>>>
>>> On Mon, Nov 18, 2013 at 3:59 PM, Stephen Sprague <[EMAIL PROTECTED]>wrote:
>>>
>>>> A word of warning for users of HiveServer2 - version 0.11 at least.
>>>> This puppy has the ability crash and/or hang your server with a memory leak.
>>>>
>>>> Apparently its not new since googling shows this discussed before and i
>>>> see reference to a workaround here:
>>>>
>>>> https://cwiki.apache.org/confluence/display/Hive/Setting+up+HiveServer2
>>>>
>>>> Anyhoo. Consider this a Public Service Announcement. Take heed.
>>>>
>>>> Regards,
>>>> Stephen.
>>>>
>>>>
>>>>
>>>>
>>>
>>
>>
>> --
>> ----------------------------------------------------------
>> Good judgement comes with experience.
>> Experience comes with bad judgement.
>> ----------------------------------------------------------
>> Roberto Congiu - Data Engineer - OpenX
>> tel: +1 626 466 1141
>>
>
>
--
----------------------------------------------------------
Good judgement comes with experience.
Experience comes with bad judgement.
----------------------------------------------------------
Roberto Congiu - Data Engineer - OpenX
tel: +1 626 466 1141
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB