Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro, mail # dev - Thoughts on an RPC protocol


+
Bruce Mitchener 2010-04-07, 22:13
Copy link to this message
-
Re: Thoughts on an RPC protocol
Bruce Mitchener 2010-04-08, 15:54
While I recommend actually reading RFC 3080 (it is an easy read), this
summary may help...

Framing: Length prefixed data, nothing unusual.
Encoding: Messages are effectively this:

enum message_type {
    message,            // a request
    reply,                  // when there's only a single reply
    answer,               // when there are multiple replies, send multiple
answers and then a null.
    null,                    // terminate a chain of replies
    error,                  // oops, there was an error
}

struct message {
    enum message_type message_type;
    int channel;
    int message_id;
    bool more;          // Is this message complete, or is more data coming?
for streaming
    int sequence_number; // see RFC 3080
    optional int answer_number; // Used for answers
    bytes payload;   // The actual RPC command, still serialized here
}

When a connection is opened, there's initially one channel, channel 0. That
channel is used for commands controlling the connection state, like opening
and closing channels.  We should also perform Avro RPC handshakes over
channel 0.

Channels allow for concurrency.  You can send requests/messages down
multiple channels and process them independently. Messages on a single
channel need to be processed in order though. This allows for both
guaranteed order of execution (within a single channel) and greater
concurrency (multiple channels).

Streaming happens in 2 ways.

The first way is to flip the more flag on a message. This means that the
data has been broken up over multiple messages and you need to receive the
whole thing before processing it.

The second is to have multiple answers (followed by a null frame) to a
single request message.  This allows you to process the data in a streaming
fashion.  The only thing that this doesn't allow is to process the data
being sent in a streaming fashion, but you could look at doing that by
sending multiple request messages instead.

Security and privacy can be handled by SASL.

The RFC defines a number of ways in which you can detect buggy
implementations of the protocol or invalid data being sent (framing /
encoding violations).

This should be pretty straight forward to implement, and as such (and since
I need such a thing in the immediate future), I've already begun an
implementation in C.

 - Bruce

On Wed, Apr 7, 2010 at 4:13 PM, Bruce Mitchener
<[EMAIL PROTECTED]>wrote:

> I'm assuming that the goals of an optimized transport for Avro RPC are
> something like the following:
>
>  * Framing should be efficient, easy to implement.
>  * Streaming of large values, both as part of a request and as a response
> is very important.
>  * Being able to have multiple concurrent requests in flight, while also
> being able to have ordering guarantees where desired is necessary.
>  * It should be easy to implement this in Java, C, Python, Ruby, etc.
>  * Security is or will be important. This security can include
> authorization as well as privacy concerns.
>
> I'd like to see something based largely upon RFC 3080, with some
> simplifications and extensions:
>
>     http://www.faqs.org/rfcs/rfc3080.html
>
> What does this get us?
>
>  * This system has mechanisms in place for streaming both a single large
> message and breaking a single reply up into multiple answers, allowing for
> pretty flexible streaming.  (You can even mix these by having an answer that
> gets chunked itself.)
>  * Concurrency is achieved by having multiple channels. Each channel
> executes messages in order, so you have a good mechanism for sending
> multiple things at once as well as maintaining ordering guarantees as
> necessary.
>  * Reporting errors is very clear as it is a separate response type.
>  * It has already been specified pretty clearly and we'd just be evolving
> that to something that more closely matches our needs.
>  * It specifies sufficient data that you could implement this over
> transports other than TCP, such as UDP.
>
> Changes, rough list:
+
Bo Shi 2010-04-08, 21:49
+
Bruce Mitchener 2010-04-08, 22:19
+
Doug Cutting 2010-04-08, 22:43
+
Jeremy Custenborder 2010-04-09, 01:29
+
Bruce Mitchener 2010-04-09, 06:35
+
Scott Carey 2010-04-09, 16:00
+
Bruce Mitchener 2010-04-09, 16:16
+
Bo Shi 2010-04-09, 18:56
+
Scott Carey 2010-04-09, 20:54
+
Doug Cutting 2010-04-09, 21:29
+
Jeff Hodges 2010-04-10, 17:48
+
Jeff Hodges 2010-04-10, 17:53
+
Jeff Hodges 2010-04-10, 17:57
+
Bruce Mitchener 2010-04-10, 17:59
+
Jeff Hodges 2010-04-10, 18:08
+
Doug Cutting 2010-04-12, 20:46
+
Jeff Hodges 2010-04-12, 21:48
+
Jeff Hodges 2010-04-12, 21:58