Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Utility of client gateways vs client libraries connecting directly to region servers & zookeeper ?


Copy link to this message
-
Re: Utility of client gateways vs client libraries connecting directly to region servers & zookeeper ?
For thrift, there is already such support.

Take a look at (0.94 codebase):
src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java

 * HRegionThriftServer - this class starts up a Thrift server in the same
 * JVM where the RegionServer is running. It inherits most of the
 * functionality from the standard ThriftServer.

Cheers

On Wed, Jun 5, 2013 at 6:03 AM, Simon Majou <[EMAIL PROTECTED]> wrote:

> Hello,
>
> I wonder, why spawing an extra process for the different gateways (thrift,
> stargate, avoc ...), when you can make your app discuss directly with
> region servers (and zookeeper) ?
>
> If your app is in Java you can use the native client so your app
> communicates directly with the region servers. That's great for Java, but
> in all other languages you have to go through an additional gateway.
>
> This additional gateway means doubling the burden in IO, and an extra step
> of parsing (eg more CPU used & more latency).
>
> Why not adding directly the REST / thrift / avoc interfaces directly into
> the region servers ?
>
> From a development point of view it would be easier as you can use directly
> the API of the region servers.
>
> From a load point of view, all the requests sent at those gateways go to
> some region server afterward, so adding gateways really doesn't help.
>
> Of course there is the cost of parsing data, but it is probably quite the
> same as the current parsing. And you can spawn additional processes
> directly from Java easily.
>
> (BTW it would be great also to use some evented frameword (like
> http://vertx.io/) to avoid wasting resource in IO)
>
> Simon
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB