Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> Avro at Flurry


+
Anthony Watkins 2012-07-12, 20:11
Copy link to this message
-
Re: Avro at Flurry
On Thu, Jul 12, 2012 at 1:11 PM, Anthony Watkins <[EMAIL PROTECTED]> wrote:
> http://tech.flurry.com/apache-avro-at-flurry.

Nice article!  Thanks for writing it.

Your idea of using different endpoints for different versions of the
protocol is an interesting one.  For example, a protocol's fingerprint
might be part of the URL path where requests are made.   If I
understand correctly, this requires that an endpoint exists for the
exact version of the protocol that every client uses.  As you upgrade
clients, the endpoints for older versions must be kept until all
clients are upgraded.  Is that right?

Note that if the server already has the client's protocol version,
then Avro's standard handshake mechanism does not transmit the
protocol, only hashes of it.  Even when protocols do not match they're
only transmitted by the first client to connect to the server with
that version.  After that the server caches it.  However the handshake
does add ~32 bytes to each connection, for the MD5 of both the
client's and the servers protocol.

The Java handshake implementation could be made more extensible.
Currently the server is initially only aware of one version of the
protocol, its own.  But it could instead be started with a set of
known protocol versions that clients might use.  If a client connects
using a known, compatible version then the server could accept it and
also use the client's version to serialize responses, avoiding the
transmission of any protocols.  If this sounds useful to someone,
please file an issue in Jira.

Doug
+
Anthony Watkins 2012-07-12, 22:01
+
Doug Cutting 2012-07-12, 22:20
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB