Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # dev >> Thoughts on an RPC protocol


Copy link to this message
-
Re: Thoughts on an RPC protocol
Doug,

I'm happy to hear that you like this approach!

Allocation of channels seems to be something specific to an application.  In
my app, I'd have a channel for the streaming data that is constantly
arriving and a channel for making requests on and getting back answers
immediately.  Others could have a channel per object or whatever.

Are your proxy servers custom software or are they just passing traffic
along directly? If they're Avro-aware, then they can manage the handshaking
process when routing to a new peer.  Is this something that is actively
happening today or just something that is possible?

I definitely agree about not wanting a handshake per request. For my
application that would add a lot of overhead in terms of the data
transmitted.  (I'm sending a lot of small requests, hopefully many thousands
per second...)  I would be much much happier being able to have a handshake
per connection (or per channel open).

 - Bruce

On Thu, Apr 8, 2010 at 4:43 PM, Doug Cutting <[EMAIL PROTECTED]> wrote:

> Bruce,
>
> Overall this looks like a good approach to me.
>
> How do you anticipate allocating channels?  I'm guessing this would be one
> per client object, that a pool of open connections to servers would be
> maintained, and creating a new client object would allocate a new channel.
>
> Currently we perform a handshake per request.  This is fairly cheap and
> permits things like routing through proxy servers.  Different requests over
> the same connection can talk to different backend servers running different
> versions of the protocol.  Also consider the case where, between calls on an
> object, the connection times out, and a new session is established and a new
> handshake must take place.
>
> That said, having a session where the handshake can be assumed vastly
> simplifies one-way messages.  Without a response or error on which to prefix
> a handshake response, a one-way client has no means to know that the server
> was able to even parse its request.  Yet we'd still like a handshake for
> one-way messages, so that clients and servers need not be versioned in
> lockstep.  So the handshake-per-request model doesn't serve one-way messages
> well.
>
> How can we address both of these needs: to permit flexible payload routing
> and efficient one-way messaging?
>
> Doug
>
>
> Bruce Mitchener wrote:
>
>>  * Think about adding something for true one-way messages, but an empty
>> reply frame is probably sufficient, since that still allows reporting
>> errors
>> if needed (or desired).
>>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB