Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # dev >> Avro RPC interop


Copy link to this message
-
Re: Avro RPC interop
A proliferation of RPC methods would create interoperability problems.
So my idea is that Avro should specify just two standard RPC methods:
  - HTTP; and
  - a single alternative that permits higher-performance, secure
implementations.

We've yet to settle on the second.  In particular, we need to agree on
how to implement:
 - out of order responses; and
 - authentication and encryption.

For out-of-order responses I made a proposal in AVRO-625 that we add an
optional call ID to request metadata.  The Netty implementation instead
puts a call ID outside of the framing.  I can see arguments for either
approach and don't have a strong preference.  We primarily need to
choose one and implement it consistently.

For authentication and encryption I have proposed SASL.  SASL's
anonymous mechanism is easy to implement, by just placing a fixed string
at the front of the first request and response (see the "Anonymous
Mechanism" section of http://avro.apache.org/docs/current/sasl.html).
With SASL, implementations can optionally negotiate authentication and
encryption.

I'd love to see us get this hashed out and included in 1.7.  I'd like to
see us have at least two implementations before we declare it a standard
part of the specification.  In Java we could add SASL support to
NettyTransceiver and/or add out-of-order support to SaslTransceiver.
Then we'd need someone to implement at least one other compatible
implementation (Python?, C?).

Doug

On 03/04/2012 03:56 PM, Mike Percy wrote:
> Hi Avro devs,
> What is the latest on Avro RPC interoperability? Flume 1.x is currently using the Avro Netty implementation as an RPC protocol, however I am not aware of compatible implementations of this transport in languages other than Java [1] and I wonder if this protocol will ever be compatible [2] with a future standard [3].
>
> Would any of you folks recommend running RPC APIs intended for public consumption on top of NettyServer at this time? It seems to be the one with the most active development going on, and appears to have significant interest, which is a good sign. Since there are other RPC transports available as well - SaslSocketServer and HttpServer - I am also wondering what anecdotal experience others can share about these implementations, in terms of speed, interoperability, and maturity.
>
> Thanks in advance for your thoughts.
>
> Regards,
> Mike
>
> [1] https://mail-archives.apache.org/mod_mbox/avro-dev/201201.mbox/%[EMAIL PROTECTED]%3E
> [2] https://mail-archives.apache.org/mod_mbox/avro-dev/201201.mbox/%[EMAIL PROTECTED]%3E
> [3] https://issues.apache.org/jira/browse/AVRO-341 - specify avro transport in spec
>