Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # dev >> Thoughts on an RPC protocol

Bruce Mitchener 2010-04-07, 22:13
Bruce Mitchener 2010-04-08, 15:54
Bo Shi 2010-04-08, 21:49
Bruce Mitchener 2010-04-08, 22:19
Doug Cutting 2010-04-08, 22:43
Jeremy Custenborder 2010-04-09, 01:29
Copy link to this message
Re: Thoughts on an RPC protocol

I'm happy to hear that you like this approach!

Allocation of channels seems to be something specific to an application.  In
my app, I'd have a channel for the streaming data that is constantly
arriving and a channel for making requests on and getting back answers
immediately.  Others could have a channel per object or whatever.

Are your proxy servers custom software or are they just passing traffic
along directly? If they're Avro-aware, then they can manage the handshaking
process when routing to a new peer.  Is this something that is actively
happening today or just something that is possible?

I definitely agree about not wanting a handshake per request. For my
application that would add a lot of overhead in terms of the data
transmitted.  (I'm sending a lot of small requests, hopefully many thousands
per second...)  I would be much much happier being able to have a handshake
per connection (or per channel open).

 - Bruce

On Thu, Apr 8, 2010 at 4:43 PM, Doug Cutting <[EMAIL PROTECTED]> wrote:

> Bruce,
> Overall this looks like a good approach to me.
> How do you anticipate allocating channels?  I'm guessing this would be one
> per client object, that a pool of open connections to servers would be
> maintained, and creating a new client object would allocate a new channel.
> Currently we perform a handshake per request.  This is fairly cheap and
> permits things like routing through proxy servers.  Different requests over
> the same connection can talk to different backend servers running different
> versions of the protocol.  Also consider the case where, between calls on an
> object, the connection times out, and a new session is established and a new
> handshake must take place.
> That said, having a session where the handshake can be assumed vastly
> simplifies one-way messages.  Without a response or error on which to prefix
> a handshake response, a one-way client has no means to know that the server
> was able to even parse its request.  Yet we'd still like a handshake for
> one-way messages, so that clients and servers need not be versioned in
> lockstep.  So the handshake-per-request model doesn't serve one-way messages
> well.
> How can we address both of these needs: to permit flexible payload routing
> and efficient one-way messaging?
> Doug
> Bruce Mitchener wrote:
>>  * Think about adding something for true one-way messages, but an empty
>> reply frame is probably sufficient, since that still allows reporting
>> errors
>> if needed (or desired).
Scott Carey 2010-04-09, 16:00
Bruce Mitchener 2010-04-09, 16:16
Bo Shi 2010-04-09, 18:56
Scott Carey 2010-04-09, 20:54
Doug Cutting 2010-04-09, 21:29
Jeff Hodges 2010-04-10, 17:48
Jeff Hodges 2010-04-10, 17:53
Jeff Hodges 2010-04-10, 17:57
Bruce Mitchener 2010-04-10, 17:59
Jeff Hodges 2010-04-10, 18:08
Doug Cutting 2010-04-12, 20:46
Jeff Hodges 2010-04-12, 21:48
Jeff Hodges 2010-04-12, 21:58