Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro, mail # dev - Thoughts on an RPC protocol


+
Bruce Mitchener 2010-04-07, 22:13
+
Bruce Mitchener 2010-04-08, 15:54
+
Bo Shi 2010-04-08, 21:49
+
Bruce Mitchener 2010-04-08, 22:19
+
Doug Cutting 2010-04-08, 22:43
+
Jeremy Custenborder 2010-04-09, 01:29
+
Bruce Mitchener 2010-04-09, 06:35
+
Scott Carey 2010-04-09, 16:00
Copy link to this message
-
Re: Thoughts on an RPC protocol
Bruce Mitchener 2010-04-09, 16:16
On Fri, Apr 9, 2010 at 10:00 AM, Scott Carey <[EMAIL PROTECTED]>wrote:

>
> On Apr 8, 2010, at 11:35 PM, Bruce Mitchener wrote:
>
> > Doug,
> >
> > I'm happy to hear that you like this approach!
> >
> > Allocation of channels seems to be something specific to an application.
>  In
> > my app, I'd have a channel for the streaming data that is constantly
> > arriving and a channel for making requests on and getting back answers
> > immediately.  Others could have a channel per object or whatever.
>
> If this is all on one TCP port, then channels will interfere with one
> another somewhat -- the transport layer will see packets arrive in the order
> they were sent.  If one packet in your streaming data stalls, both channels
> will stall.  Depending on the application requirements, this might be fine.
>  But it should be made clear that channels are not independent, they are
> just interleaved over one ordered data stream.  How each implementation
> orders sending data on one end will affect order on the other side.
Agreed, that's just a fact of life with TCP.  Perhaps if SCTP ever gets some
traction, then people can do a mapping for that.  In the meantime, we could
look at what RFC 3081 did in the TCP mapping for RFC 3080 with respect to
flow control.

> I definitely agree about not wanting a handshake per request. For my
> > application that would add a lot of overhead in terms of the data
> > transmitted.  (I'm sending a lot of small requests, hopefully many
> thousands
> > per second...)  I would be much much happier being able to have a
> handshake
> > per connection (or per channel open).
> >
>
> Handshake per request will limit WAN usage.  Doubling request latency isn't
> a problem for local networks with sub 0.1ms RTTs, but it is a problem with
> 25ms RTTs.  Round trips aren't free on the processing or bandwidth side
> either.   If there is a way to meet most goals and limit extra handshakes to
> specific cases that would be a significant performance improvement.
We agree very strongly here.

 - Bruce
+
Bo Shi 2010-04-09, 18:56
+
Scott Carey 2010-04-09, 20:54
+
Doug Cutting 2010-04-09, 21:29
+
Jeff Hodges 2010-04-10, 17:48
+
Jeff Hodges 2010-04-10, 17:53
+
Jeff Hodges 2010-04-10, 17:57
+
Bruce Mitchener 2010-04-10, 17:59
+
Jeff Hodges 2010-04-10, 18:08
+
Doug Cutting 2010-04-12, 20:46
+
Jeff Hodges 2010-04-12, 21:48
+
Jeff Hodges 2010-04-12, 21:58