Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # dev >> Thoughts on an RPC protocol

Copy link to this message
Re: Thoughts on an RPC protocol
On Fri, Apr 9, 2010 at 10:00 AM, Scott Carey <[EMAIL PROTECTED]>wrote:

> On Apr 8, 2010, at 11:35 PM, Bruce Mitchener wrote:
> > Doug,
> >
> > I'm happy to hear that you like this approach!
> >
> > Allocation of channels seems to be something specific to an application.
>  In
> > my app, I'd have a channel for the streaming data that is constantly
> > arriving and a channel for making requests on and getting back answers
> > immediately.  Others could have a channel per object or whatever.
> If this is all on one TCP port, then channels will interfere with one
> another somewhat -- the transport layer will see packets arrive in the order
> they were sent.  If one packet in your streaming data stalls, both channels
> will stall.  Depending on the application requirements, this might be fine.
>  But it should be made clear that channels are not independent, they are
> just interleaved over one ordered data stream.  How each implementation
> orders sending data on one end will affect order on the other side.
Agreed, that's just a fact of life with TCP.  Perhaps if SCTP ever gets some
traction, then people can do a mapping for that.  In the meantime, we could
look at what RFC 3081 did in the TCP mapping for RFC 3080 with respect to
flow control.

> I definitely agree about not wanting a handshake per request. For my
> > application that would add a lot of overhead in terms of the data
> > transmitted.  (I'm sending a lot of small requests, hopefully many
> thousands
> > per second...)  I would be much much happier being able to have a
> handshake
> > per connection (or per channel open).
> >
> Handshake per request will limit WAN usage.  Doubling request latency isn't
> a problem for local networks with sub 0.1ms RTTs, but it is a problem with
> 25ms RTTs.  Round trips aren't free on the processing or bandwidth side
> either.   If there is a way to meet most goals and limit extra handshakes to
> specific cases that would be a significant performance improvement.
We agree very strongly here.

 - Bruce