Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Utility of client gateways vs client libraries connecting directly to region servers & zookeeper ?


Copy link to this message
-
Re: Utility of client gateways vs client libraries connecting directly to region servers & zookeeper ?
For thrift, there is already such support.

Take a look at (0.94 codebase):
src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java

 * HRegionThriftServer - this class starts up a Thrift server in the same
 * JVM where the RegionServer is running. It inherits most of the
 * functionality from the standard ThriftServer.

Cheers

On Wed, Jun 5, 2013 at 6:03 AM, Simon Majou <[EMAIL PROTECTED]> wrote:

> Hello,
>
> I wonder, why spawing an extra process for the different gateways (thrift,
> stargate, avoc ...), when you can make your app discuss directly with
> region servers (and zookeeper) ?
>
> If your app is in Java you can use the native client so your app
> communicates directly with the region servers. That's great for Java, but
> in all other languages you have to go through an additional gateway.
>
> This additional gateway means doubling the burden in IO, and an extra step
> of parsing (eg more CPU used & more latency).
>
> Why not adding directly the REST / thrift / avoc interfaces directly into
> the region servers ?
>
> From a development point of view it would be easier as you can use directly
> the API of the region servers.
>
> From a load point of view, all the requests sent at those gateways go to
> some region server afterward, so adding gateways really doesn't help.
>
> Of course there is the cost of parsing data, but it is probably quite the
> same as the current parsing. And you can spawn additional processes
> directly from Java easily.
>
> (BTW it would be great also to use some evented frameword (like
> http://vertx.io/) to avoid wasting resource in IO)
>
> Simon
>