Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro, mail # user - Avro enhancement: asynchronous RPCs for Java clients


Copy link to this message
-
Avro enhancement: asynchronous RPCs for Java clients
James Baldassari 2011-05-31, 07:08
Hi,

I recently started playing with Avro RPCs, and while it's great that Avro
can use Netty for asynchronous I/O, all client-facing Java RPC interfaces
are currently synchronous (as far as I can tell).  It would be nice to have
asynchronous RPCs that use callbacks to take full advantage of Netty's
asynchronous features.  I found at least one other request for this feature
on the Avro list (http://bit.ly/iCD0ae), and I believe this enhancement is
already documented as AVRO-539.

I took it on as a weekend project to add async Java RPCs to Avro, and I
think I have it all working now including unit tests.  I'd love to
contribute this patch once I've gotten some feedback, cleaned up the
documentation, and written a few more tests.  I'll give a quick example
demonstrating this new functionality.  Consider the following IDL and its
associated schema which use the asynchronous feature:

IDL:
protocol Calculator {
  int add(int arg1, int arg2) async;
}

Schema:
{"protocol":"Calculator","messages":{
  "add":{
    "request":[{"name":"arg1","type":"int"},{"name":"arg2","type":"int"}],
    "response":"int",
    "async":true}}}

When the "async" keyword/property is present in a message, the Avro Java
compiler generates two methods instead of one: the standard synchronous
method and an asynchronous companion method.  For the example I gave above,
the following interface would be generated (the static PROTOCOL field and
package names are omitted for brevity):

public interface Calculator {
  int add(int arg1, int arg2) throws AvroRemoteException;
  void addAsync(int arg1, int arg2, Callback<Integer> callback) throws
IOException;
}

The addAsync method returns immediately without blocking (except one special
case which I'll describe shortly).  The callback provided as the last
argument to addAsync will be invoked with the result of the RPC as soon as
the server returns it.  There are a couple of caveats.  The first RPC,
whether synchronous or asynchronous, must block until the client-server
handshake has been completed.  Subsequent async RPCs will never block.
Also, only the NettyTransceiver currently works with async RPCs; all other
transceivers throw UnsupportedOperationException.

So that's the basic overview of my changes.  Please let me know if there are
any comments or questions.  Is this something people are interested in?  If
so, should I post the patch in AVRO-539 or somewhere else?

Thanks,
James