Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro, mail # dev - Thoughts on an RPC protocol


+
Bruce Mitchener 2010-04-07, 22:13
+
Bruce Mitchener 2010-04-08, 15:54
+
Bo Shi 2010-04-08, 21:49
+
Bruce Mitchener 2010-04-08, 22:19
+
Doug Cutting 2010-04-08, 22:43
+
Jeremy Custenborder 2010-04-09, 01:29
Copy link to this message
-
Re: Thoughts on an RPC protocol
Bruce Mitchener 2010-04-09, 06:35
Doug,

I'm happy to hear that you like this approach!

Allocation of channels seems to be something specific to an application.  In
my app, I'd have a channel for the streaming data that is constantly
arriving and a channel for making requests on and getting back answers
immediately.  Others could have a channel per object or whatever.

Are your proxy servers custom software or are they just passing traffic
along directly? If they're Avro-aware, then they can manage the handshaking
process when routing to a new peer.  Is this something that is actively
happening today or just something that is possible?

I definitely agree about not wanting a handshake per request. For my
application that would add a lot of overhead in terms of the data
transmitted.  (I'm sending a lot of small requests, hopefully many thousands
per second...)  I would be much much happier being able to have a handshake
per connection (or per channel open).

 - Bruce

On Thu, Apr 8, 2010 at 4:43 PM, Doug Cutting <[EMAIL PROTECTED]> wrote:

> Bruce,
>
> Overall this looks like a good approach to me.
>
> How do you anticipate allocating channels?  I'm guessing this would be one
> per client object, that a pool of open connections to servers would be
> maintained, and creating a new client object would allocate a new channel.
>
> Currently we perform a handshake per request.  This is fairly cheap and
> permits things like routing through proxy servers.  Different requests over
> the same connection can talk to different backend servers running different
> versions of the protocol.  Also consider the case where, between calls on an
> object, the connection times out, and a new session is established and a new
> handshake must take place.
>
> That said, having a session where the handshake can be assumed vastly
> simplifies one-way messages.  Without a response or error on which to prefix
> a handshake response, a one-way client has no means to know that the server
> was able to even parse its request.  Yet we'd still like a handshake for
> one-way messages, so that clients and servers need not be versioned in
> lockstep.  So the handshake-per-request model doesn't serve one-way messages
> well.
>
> How can we address both of these needs: to permit flexible payload routing
> and efficient one-way messaging?
>
> Doug
>
>
> Bruce Mitchener wrote:
>
>>  * Think about adding something for true one-way messages, but an empty
>> reply frame is probably sufficient, since that still allows reporting
>> errors
>> if needed (or desired).
>>
>
+
Scott Carey 2010-04-09, 16:00
+
Bruce Mitchener 2010-04-09, 16:16
+
Bo Shi 2010-04-09, 18:56
+
Scott Carey 2010-04-09, 20:54
+
Doug Cutting 2010-04-09, 21:29
+
Jeff Hodges 2010-04-10, 17:48
+
Jeff Hodges 2010-04-10, 17:53
+
Jeff Hodges 2010-04-10, 17:57
+
Bruce Mitchener 2010-04-10, 17:59
+
Jeff Hodges 2010-04-10, 18:08
+
Doug Cutting 2010-04-12, 20:46
+
Jeff Hodges 2010-04-12, 21:48
+
Jeff Hodges 2010-04-12, 21:58