Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro, mail # user - Avro at Flurry


Copy link to this message
-
Re: Avro at Flurry
Doug Cutting 2012-07-12, 20:52
On Thu, Jul 12, 2012 at 1:11 PM, Anthony Watkins <[EMAIL PROTECTED]> wrote:
> http://tech.flurry.com/apache-avro-at-flurry.

Nice article!  Thanks for writing it.

Your idea of using different endpoints for different versions of the
protocol is an interesting one.  For example, a protocol's fingerprint
might be part of the URL path where requests are made.   If I
understand correctly, this requires that an endpoint exists for the
exact version of the protocol that every client uses.  As you upgrade
clients, the endpoints for older versions must be kept until all
clients are upgraded.  Is that right?

Note that if the server already has the client's protocol version,
then Avro's standard handshake mechanism does not transmit the
protocol, only hashes of it.  Even when protocols do not match they're
only transmitted by the first client to connect to the server with
that version.  After that the server caches it.  However the handshake
does add ~32 bytes to each connection, for the MD5 of both the
client's and the servers protocol.

The Java handshake implementation could be made more extensible.
Currently the server is initially only aware of one version of the
protocol, its own.  But it could instead be started with a set of
known protocol versions that clients might use.  If a client connects
using a known, compatible version then the server could accept it and
also use the client's version to serialize responses, avoiding the
transmission of any protocols.  If this sounds useful to someone,
please file an issue in Jira.

Doug