Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> Avro enhancement: asynchronous RPCs for Java clients

Copy link to this message
Re: Avro enhancement: asynchronous RPCs for Java clients
Hi James,

I think this is a fine approach.  You're correct that the place to do it is
in the code generator.  It's not obvious to me that it belongs in the
schema, however, since you might have two different pieces of software that
want to use the same RPCs differently.  Is there any harm in
always generating both types?

As for your API, you'll want to specify very explicitly what happens to the
exceptions that an Avro RPC call may declare.  Future<V> is one mechanism.
 As is your Callback<Foo> mechanism, if there's a way to get at exceptional

On Tue, May 31, 2011 at 12:08 AM, James Baldassari <[EMAIL PROTECTED]>wrote:

> Hi,
> I recently started playing with Avro RPCs, and while it's great that Avro
> can use Netty for asynchronous I/O, all client-facing Java RPC interfaces
> are currently synchronous (as far as I can tell).  It would be nice to have
> asynchronous RPCs that use callbacks to take full advantage of Netty's
> asynchronous features.  I found at least one other request for this feature
> on the Avro list (http://bit.ly/iCD0ae), and I believe this enhancement is
> already documented as AVRO-539.
> I took it on as a weekend project to add async Java RPCs to Avro, and I
> think I have it all working now including unit tests.  I'd love to
> contribute this patch once I've gotten some feedback, cleaned up the
> documentation, and written a few more tests.  I'll give a quick example
> demonstrating this new functionality.  Consider the following IDL and its
> associated schema which use the asynchronous feature:
> IDL:
> protocol Calculator {
>   int add(int arg1, int arg2) async;
> }
> Schema:
> {"protocol":"Calculator","messages":{
>   "add":{
>     "request":[{"name":"arg1","type":"int"},{"name":"arg2","type":"int"}],
>     "response":"int",
>     "async":true}}}
> When the "async" keyword/property is present in a message, the Avro Java
> compiler generates two methods instead of one: the standard synchronous
> method and an asynchronous companion method.  For the example I gave above,
> the following interface would be generated (the static PROTOCOL field and
> package names are omitted for brevity):
> public interface Calculator {
>   int add(int arg1, int arg2) throws AvroRemoteException;
>   void addAsync(int arg1, int arg2, Callback<Integer> callback) throws
> IOException;
> }
> The addAsync method returns immediately without blocking (except one
> special case which I'll describe shortly).  The callback provided as the
> last argument to addAsync will be invoked with the result of the RPC as soon
> as the server returns it.  There are a couple of caveats.  The first RPC,
> whether synchronous or asynchronous, must block until the client-server
> handshake has been completed.  Subsequent async RPCs will never block.
> Also, only the NettyTransceiver currently works with async RPCs; all other
> transceivers throw UnsupportedOperationException.
> So that's the basic overview of my changes.  Please let me know if there
> are any comments or questions.  Is this something people are interested in?
> If so, should I post the patch in AVRO-539 or somewhere else?
> Thanks,
> James