Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro, mail # user - Writing Unsolicited Messages to a Connected Netty Client


Copy link to this message
-
Re: Writing Unsolicited Messages to a Connected Netty Client
James Baldassari 2012-01-20, 18:15
Hi Armin,

First I'd like to explain why the server-initiated messages are
problematic.  Allowing the server to send unsolicited messages back to the
client may work for some Transceiver implementations (possibly PHP), but
this change would not be compatible with NettyTransceiver.  When the
NettyTransceiver receives a message from the server, it needs to know which
callback to invoke in order to pass the message back correctly to the
client.  There could be several RPCs "in flight" concurrently, so one of
NettyTransceiver's jobs is to match up the response with the request that
initiated it.  If the client didn't initiate the RPC then NettyTransceiver
won't know where to deliver the message, unless there were some catch-all
callback that would be invoked whenever one of these "unsolicited" messages
were received.  So although you're probably only interested in the PHP
client, allowing the server to send these unsolicited messages would
potentially break NettyTransceiver (and possibly other implementations as
well).

Shaun's idea of having the client poll the server periodically would
definitely work.  What we want to do is have the client receive
notifications from the server as they become available on the server side,
but we also don't want the client to be polling with such a high frequency
that a lot of CPU and bandwidth resources are wasted.  I think we can get
the best of both worlds by copying the Comet pattern, i.e. the "long poll"
but using the Avro RPC layer instead of (or on top of) HTTP.  First we'll
start with Shaun's update listener interface:

protocol WeatherUpdateListener {
  WeatherUpdate listenForUpdate();
}

The PHP client would invoke this RPC against the server in a tight loop.
On the server side, the RPC will block until there is an update that is
ready to be sent to the client.  When the client does receive an event from
the server (or some timeout occurs), the client will immediately send
another poll to the server and block until the next update is received.  In
this way the client will not be flooding the server with RPCs, but the
client will also get updates in a timely manner.

See the following for more info about Comet:
http://www.javaworld.com/javaworld/jw-03-2008/jw-03-asynchhttp.html?page=6

-James
On Fri, Jan 20, 2012 at 12:44 PM, Armin Garcia <[EMAIL PROTECTED]>wrote:

> Hi Shaun,
>
> This is definitely another way.  I share your same concern.  I have to
> keep an eye out for high availablilty and high throughput.  I'll be
> depending on this connection to support a massive amount of data.
>
>                -Armin
>
>
> On Fri, Jan 20, 2012 at 9:25 AM, Shaun Williams <[EMAIL PROTECTED]>wrote:
>
>> Another solution is to use the response leg of a transaction to push
>> messages to the client, e.g. provide a server protocol like this:
>>
>> WeatherUpdate listenForUpdate();
>>
>> This would essentially block until an update is available.  The only
>> problem is that if the client is expecting a series of updates, it would
>> need to call this method again after receiving each update.
>>
>> This is not an ideal solution, but it might solve your problem.
>>
>> -Shaun
>>
>>
>>
>> On Jan 20, 2012, at 8:24 AM, Armin Garcia wrote:
>>
>> Hi James,
>>
>>
>> First, thank you for your response.
>>
>>
>> Yes, you are right.  I am trying to setup a bi-directional communication
>> link.  Your suggestion would definitely accomplish this requirement.  I was
>> hoping the same channel could be reused without having to establish another
>> uni-directional link.  Netty or rather NIO is inherently bi-directional.  I
>> am suspecting RPC by definition is only uni-directional or rather a pull
>> technology?
>>
>>
>> One of my goals is to support as many different language bindings using
>> Avro.  PHP is one of those languages.  Unfortunately, the PHP library can
>> only function as a client.
>>
>>
>>                -Armin
>>
>> On Fri, Jan 20, 2012 at 7:47 AM, James Baldassari <[EMAIL PROTECTED]>wrote: