Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # dev >> Thoughts on an RPC protocol


Copy link to this message
-
Re: Thoughts on an RPC protocol
On Fri, Apr 9, 2010 at 10:00 AM, Scott Carey <[EMAIL PROTECTED]>wrote:

>
> On Apr 8, 2010, at 11:35 PM, Bruce Mitchener wrote:
>
> > Doug,
> >
> > I'm happy to hear that you like this approach!
> >
> > Allocation of channels seems to be something specific to an application.
>  In
> > my app, I'd have a channel for the streaming data that is constantly
> > arriving and a channel for making requests on and getting back answers
> > immediately.  Others could have a channel per object or whatever.
>
> If this is all on one TCP port, then channels will interfere with one
> another somewhat -- the transport layer will see packets arrive in the order
> they were sent.  If one packet in your streaming data stalls, both channels
> will stall.  Depending on the application requirements, this might be fine.
>  But it should be made clear that channels are not independent, they are
> just interleaved over one ordered data stream.  How each implementation
> orders sending data on one end will affect order on the other side.
Agreed, that's just a fact of life with TCP.  Perhaps if SCTP ever gets some
traction, then people can do a mapping for that.  In the meantime, we could
look at what RFC 3081 did in the TCP mapping for RFC 3080 with respect to
flow control.

> I definitely agree about not wanting a handshake per request. For my
> > application that would add a lot of overhead in terms of the data
> > transmitted.  (I'm sending a lot of small requests, hopefully many
> thousands
> > per second...)  I would be much much happier being able to have a
> handshake
> > per connection (or per channel open).
> >
>
> Handshake per request will limit WAN usage.  Doubling request latency isn't
> a problem for local networks with sub 0.1ms RTTs, but it is a problem with
> 25ms RTTs.  Round trips aren't free on the processing or bandwidth side
> either.   If there is a way to meet most goals and limit extra handshakes to
> specific cases that would be a significant performance improvement.
We agree very strongly here.

 - Bruce
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB