Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro, mail # user - Avro Callback instance and Protocol formats


Copy link to this message
-
Re: Avro Callback instance and Protocol formats
James Baldassari 2012-02-24, 16:41
Hi Bhavani,

Regarding the protocol formats, I think you're referring to the difference
between declaring protocols in JSON and declaring them using the Avro IDL.
As far as I know there aren't any functional differences between the two
(i.e. you can do everything in one that you can do in the other), but I
prefer IDL because I find it to be much more readable.

When you generate record/protocol code using the Maven, the first step is
that the Avro Maven plugin is invoked.  If you're interested in seeing this
code, it's in the avro-maven-plugin module in the Avro source.  The Avro
Maven plugins parse the protocol and then uses the SpecificCompiler class
from the avro-compiler project  to generate the interfaces/classes for the
protocols.  The generated classes will end up in
target/generated-sources/avro by default.  Keep in mind that if you're
using an IDE like Eclipse you'll probably have to manually add this
directory to your project's source path.

The callback-enabled version of the protocols are generated as
sub-interfaces of the non-callback protocols.  For example, if you have a
protocol called Mail, then you'll have a corresponding Java interface
called Mail, and Mail will have a sub-interface called Mail.Callback.  If
your generated code has Mail but not Mail.Callback then I think you must be
using a version of Avro that does not support callbacks.  The callback API
was introduced in 1.5.2.  Which version are you using?

I realize that there isn't much documentation about how callbacks work
other than the JavaDocs.  I'm hoping I'll find the time to write some
documentation for the Avro website soon.  In the meantime, here are a few
things to keep in mind:

- The callback interface only affects the client side of the RPC protocol.
The server is always going to implement Mail, but the client has the option
to use either Mail or Mail.Callback to interact with the server.

- Only the Netty implementation of the server and transceiver currently
support asynchronous callbacks.  If you use the callback interface with any
other transceiver implementation, it will simply behave synchronously.

- The very first RPC will execute synchronously regardless of whether the
callback interface is used because the RPC handshake has to be completed,
but all subsequent RPCs using that Transceiver instance will be
asynchronous.  See https://issues.apache.org/jira/browse/AVRO-1008 for a
discussion of this.  (BTW, Doug, I know I owe you a response on this
ticket.  I'm hoping to get to it soon)

Let me know if I haven't answered any of your questions.

-James
On Fri, Feb 24, 2012 at 11:07 AM, Bhavani Ikkurthi <[EMAIL PROTECTED]> wrote:

> Hi,
>
> I am pretty much new to using Avro. I started off playing around with
> James Baldassari's github code: https://github.com/jbaldassari/Avro-RPC
> I am more interested in Avro's async RPC feature. In that process of
> creating a Server and Transceiver I encountered few troubles and have the
> below questions.
>
> 1. How does the different Message protocol formats like .avpr/.avdl differ
> from each other?
> 2. I am using .avpr file and generating the stubs using "mvn compile"
> command. Can someone please give me an insight into this compilation
> process and auto-generation process?
> 3. Proceeding further, I notice that after generating the code, I cant
> find a "Callback" interface which I assume is crucial to implementing the
> async RPC call.
>
> I am very new to Avro. So please bear with any techie-term-typos and do
> take the liberty to correct me. Any help on these questions is much
> appreciated.
>
> --
> Thanks
> Bhavani
>
>