Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # dev >> Avro governance and Avro Enhancement Proposals

Copy link to this message
Re: Avro governance and Avro Enhancement Proposals
Based on the lack of response to this, I'm left to assume that there is no
practical interest in addressing the current issues with revisions to the
specification and formalizing any structure to make these changes easier to

As such, I'll withdraw this proposal for the time being until there is
sufficient interest and energy to make such a thing worth pursuing.

 - Bruce

On Tue, Jun 8, 2010 at 9:16 AM, Bruce Mitchener

> While cutting had said that there were few revisions to the spec, there are
> at least 6 being discussed in some form right now:
>  * 3 that were in the wiki
>  * my binary RPC proposal
>  * hammer just asked for a change to the container file format to support
> checksums
>  * jmhodges (I think) and I have talked about modifying things so that you
> can refer to a type by name before defining it as long as it is defined by
> the end of the schema.
> And I'm sure that if I wanted to trawl through JIRA, the lists and various
> IRC discussions, we can find other instances.
> That leaves aside things like GenAvro, which people have talked about
> extending but isn't part of the base Avro spec.
> Following along with these things requires exposing yourself to the full
> flow of changes (from JIRA) and information (mailing list, IRC) and it isn't
> terribly easy to keep track of what's being discussed, where, and how to
> give comments.
> Whatever the result of this thread is, AEPs or not, it would be nice to
> have some way to know what changes are being proposed / discussed /
> implemented.
>  - Bruce
> On Tue, Jun 8, 2010 at 2:51 AM, Matt Massie <[EMAIL PROTECTED]> wrote:
>> On Mon, Jun 7, 2010 at 9:43 AM, Doug Cutting <[EMAIL PROTECTED]> wrote:
>> > On 06/04/2010 06:29 PM, Matt Massie wrote:
>> >
>> >> I would like to see us be a little more of a think-do-acrocy when it
>> comes
>> >> to important, high-level components of Avro (e.g. light-weight RPC).
>>  It
>> >> would be nice to have some simple consensus before code starts getting
>> >> slung
>> >> around;
>> >>
>> >
>> > I'd rather we have proofs of concept before we standardize.  I often
>> find
>> > design issues while implementing.  I like to see two working
>> implementations
>> > before we promote something as an interoperability standard.  Little
>> else of
>> > substance has gone into the spec until it's been implemented twice.
>> Of course we should have proofs of concept and I agree that you can find
>> design issues while implementing ideas.  You can also find design issues
>> by
>> having more than one person review the design before it's implemented.
>> I'm happy to drop my support for AEPs.  The main reason I supported them
>> in
>> the first place is that I felt they would be a good way of fostering
>> communication.  If the team finds no value in them, then we should drop
>> the
>> idea for now.
>> > otherwise, we'll see things in our spec like "The double is
>> >> converted into a 64-bit integer using a method equivalent to Java's
>> >> doubleToLongBits and then encoded in little-endian format."  Not to say
>> we
>> >> would ever do such a thing.  :)
>> >>
>> >
>> > I don't see a fundamental problem with that.  It's a link to clear,
>> > detailed documentation that describes a standard for serializing
>> infinities
>> > and NaNs, using zeroed mantissas.  If you'd rather make the spec more
>> > standalone, then feel free to update that passage with these details.
>>  But
>> > there's nothing Java-specific in it.  This is similar to the link to
>> > Protocol Buffers to define zig-zag encoding.  It's perhaps lazy writing,
>> but
>> > I don't think it's indicative of a technically flawed specification.  Do
>> > you?
>> >
>> I do not think it indicates a technically flawed specification.
>> -Matt