Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> HIVE Server multiple instances


Copy link to this message
-
Re: HIVE Server multiple instances
Hey Paul,

I'd be very interested in reading about your hadoop/hive setup, do you have a blog post or anything describing this setup, or some of the issues you've have with hive?

--
Matthew Rathbone
Foursquare | Software Engineer | Server Engineering Team
[EMAIL PROTECTED] | @rathboma | 4sq

On Tuesday, May 3, 2011 at 2:15 PM, Paul Ingles wrote:
HiveServer does seem to support multiple connections but I think it still has thread-safety problems (https://issues.apache.org/jira/browse/HIVE-80).
>
> We've (www.forward.co.uk) certainly had instability problems with the thrift server in the past and now run 5 or so instances behind the HAProxy load-balancer (http://haproxy.1wt.eu/). Since we did that it's been significantly better.
>
> I think the JDBC server still operates using thrift to connect to the HiveServer so I would expect it to have similar problems (but I may have got that wrong :)
>
>
> On 3 May 2011, at 18:59, Matthew Rathbone wrote:
>
> > Even if it is single threaded it certainly seems to support multiple connections.
> >
> > We run 5 workers all connected at the same time executing a different query each ( with a different connection per worker).
> >
> > Hope that helps
> >
> > Matthew
> > On Tuesday, May 3, 2011 at 1:40 PM, V.Senthil Kumar wrote:
> > Thanks Matthew. The wiki page http://wiki.apache.org/hadoop/Hive/HiveServer says
> > > its single threaded. I have a queue of queries which gets added dynamically all
> > > the time. By the time I run 1 query using 1 JDBC connection, the queue gets
> > > added more queries and builds up a backlog. So, I was that's why I was wondering
> > > whether I can run two or more instances to avoid having a big backlog in queue.
> > >
> > >
> > >
> > > ----- Original Message ----
> > > From: Matthew Rathbone <[EMAIL PROTECTED]>
> > > To: [EMAIL PROTECTED]
> > > Sent: Tue, May 3, 2011 7:46:49 AM
> > > Subject: Re: HIVE Server multiple instances
> > >
> > > Why would you want to run two? I think it is multithreaded, so you can query it
> > > from two different connections
> > >
> > > --
> > > Matthew Rathbone
> > > Foursquare | Software Engineer | Server Engineering Team
> > > [EMAIL PROTECTED] | @rathboma | 4sq
> > >
> > > On Monday, May 2, 2011 at 6:41 PM, V.Senthil Kumar wrote:
> > > Hello,
> > > >
> > > > I have one instance of HIVE JDBC server running on port 10000. Can I run
> > > > another
> > > >
> > > > instance on different port ? Would it cause a concurrency issue on the
> > > > underlying data warehouse files ? Please clarify.
> > > >
> > > > Thanks,
> > > > V.Senthil Kumar
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB