Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> Avro enhancement: asynchronous RPCs for Java clients


Copy link to this message
-
Re: Avro enhancement: asynchronous RPCs for Java clients
Thanks for the helpful feedback!

After thinking about this more, I agree that it would be cleaner and simpler
to remove the "async" keyword/property from the IDL and schema.  Instead
I'll just generate the asynchronous companion methods for all two-way
messages.

Regarding the passing of RPC results and exceptions/errors back to the
client asynchronously, I'm not sure what the best approach is.  I had
considered the use of both the future pattern and the callback pattern, but
I think the callback pattern is more useful in general.  One option I had
considered was having the async methods accept a Callback parameter and also
return a Future so that the client could choose which pattern to use on a
case-by-case basis.  I think I may go with the combined callback/future
approach as it provides clients with the most flexibility.

For Futures, error handling is straightforward: the get() method either
returns the result of the callback or throws the exception if one occurred.
The correct solution for the callback approach is less obvious, but I'll
tell you the approach that I took, and I'm open to comments.  The Callback
interface has two methods, one for handling a successful RPC result, and one
for handling exceptions that occur during RPC execution.  For each RPC,
either one or the other will be called, but never both.  The methods look
like this:

void handleCallback(T result);
void handleError(Exception error);

I'll work on making the changes you suggested, and hopefully I'll have a
patch that's in decent shape by the end of the week.

-James
On Tue, May 31, 2011 at 4:40 PM, Philip Zeyliger <[EMAIL PROTECTED]>wrote:

> Hi James,
>
> I think this is a fine approach.  You're correct that the place to do it is
> in the code generator.  It's not obvious to me that it belongs in the
> schema, however, since you might have two different pieces of software that
> want to use the same RPCs differently.  Is there any harm in
> always generating both types?
>
> As for your API, you'll want to specify very explicitly what happens to the
> exceptions that an Avro RPC call may declare.  Future<V> is one mechanism.
>  As is your Callback<Foo> mechanism, if there's a way to get at exceptional
> states.
>
>
> On Tue, May 31, 2011 at 12:08 AM, James Baldassari <[EMAIL PROTECTED]>wrote:
>
>> Hi,
>>
>> I recently started playing with Avro RPCs, and while it's great that Avro
>> can use Netty for asynchronous I/O, all client-facing Java RPC interfaces
>> are currently synchronous (as far as I can tell).  It would be nice to have
>> asynchronous RPCs that use callbacks to take full advantage of Netty's
>> asynchronous features.  I found at least one other request for this feature
>> on the Avro list (http://bit.ly/iCD0ae), and I believe this enhancement
>> is already documented as AVRO-539.
>>
>> I took it on as a weekend project to add async Java RPCs to Avro, and I
>> think I have it all working now including unit tests.  I'd love to
>> contribute this patch once I've gotten some feedback, cleaned up the
>> documentation, and written a few more tests.  I'll give a quick example
>> demonstrating this new functionality.  Consider the following IDL and its
>> associated schema which use the asynchronous feature:
>>
>> IDL:
>> protocol Calculator {
>>   int add(int arg1, int arg2) async;
>> }
>>
>> Schema:
>> {"protocol":"Calculator","messages":{
>>   "add":{
>>     "request":[{"name":"arg1","type":"int"},{"name":"arg2","type":"int"}],
>>     "response":"int",
>>     "async":true}}}
>>
>> When the "async" keyword/property is present in a message, the Avro Java
>> compiler generates two methods instead of one: the standard synchronous
>> method and an asynchronous companion method.  For the example I gave above,
>> the following interface would be generated (the static PROTOCOL field and
>> package names are omitted for brevity):
>>
>> public interface Calculator {
>>   int add(int arg1, int arg2) throws AvroRemoteException;
>>   void addAsync(int arg1, int arg2, Callback<Integer> callback) throws