Christophe Taton 2013-12-02, 21:42
Philip Zeyliger 2013-12-02, 21:56
Doug Cutting 2013-12-03, 15:20
Dan Burkert 2013-12-04, 01:28
Dan Burkert 2013-12-04, 01:41
-Re: Effort towards Avro 2.0?
Tibor Benke 2013-12-05, 08:48
In the case of RPC, I think, not just Python and Java, but all supported
languages, should be compatible with each other.
On 12/02/2013 10:56 PM, Philip Zeyliger wrote:
> It sounds like you're proposing to break language API compatibility.
> Are you also proposing to break wire compatibility for Avro HTTP RPC,
> Avro Netty RPC, and/or Avro datafiles?
> I'd be appreciative of a mechanism by which systems that happen to use
> Avro currently need not be forced to choose one version or another.
> (One approach to this is to use a different package name.)
> As for adding to your list, I'd like to see a code-generated API for
> Python. (We like to call these APIs "specific" but I find that
> terminology confusing.)
> -- Philip
> On Mon, Dec 2, 2013 at 1:42 PM, Christophe Taton
> <[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>> wrote:
> Hi all,
> Avro, in its current form, exhibits a number of limitations that
> are hard to work with or around, and hard to fix within the scope
> of Avro 1.x : fixing these issues would introduce incompatible
> changes that warrant a major version bump, ie. Avro 2.0. An Avro
> 2.0 branch would be an opportunity to address most issues that
> appeared held back for compatibility purposes so far.
> I would like to initiate an effort in this direction and I am
> willing to do the necessary work to gather and organize
> requirements, and draft a design document for what Avro 2.0 would
> look like. For this reason, if you have opinions regarding an Avro
> 2.0 branch or regarding issues and features that could fit in Avro
> 2.0, please reply to this thread.
> To bootstrap, below is a list I gathered over the last couple of
> years from several discussions:
> * Specification
> o Improved support for unions (incompatible change with
> named unions and union properties).
> o New extension data type, similar to ProtocolBuffer
> extensions (incompatible change).
> o Clear separation between Avro schema (data format) and
> specific API client concerns: for example, the way Avro
> strings are exposed through the Java API should not
> pollute the schema definition. Each particular Java client
> should configure their own decoders with the way they want
> Avro strings to be represented.
> o Clarification of compatibility and type promotion (safe
> lossless conversions vs. best-effort lossy conversions):
> promoting int to float potentially loses precision, which
> is not necessarily acceptable for all clients. Avro
> decoders should let clients configure which mode they need.
> * IDL
> o Generalized IDL for Avro schemas.
> o Support for recursive records.
> o Meta-schema : IDL definition for a schema.
> * Java API
> o Truly immutable schema objects (no properties / hashcode
> mutation after construction).
> o Immutable records.
> o Complete record builder API (current record builders do
> not play well with nested records).
> o Complete generic API (there currently is no GenericUnion
> or GenericMap).
> o Improved unions support : union values as java.lang.Object
> are less than ideal; union values could expose the union
> branch through an enum (nulls could be handled specifically).
> * Python 3 support
> * RPC
> o SASL support
> o Full Python/Java parity and interoperability.
> Please, comment or extend this list. Provided enough interest,
> I'll happily digest feedback and organize it into a document (most
> likely a wiki page?).