Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # dev >> Avro governance and Avro Enhancement Proposals

Bruce Mitchener 2010-06-03, 08:46
Philip Zeyliger 2010-06-03, 22:07
Jeff Hammerbacher 2010-06-03, 23:16
Matt Massie 2010-06-03, 23:37
Doug Cutting 2010-06-04, 00:15
Matt Massie 2010-06-05, 01:29
Doug Cutting 2010-06-07, 16:43
Matt Massie 2010-06-07, 19:51
Copy link to this message
Re: Avro governance and Avro Enhancement Proposals
While cutting had said that there were few revisions to the spec, there are
at least 6 being discussed in some form right now:

 * 3 that were in the wiki
 * my binary RPC proposal
 * hammer just asked for a change to the container file format to support
 * jmhodges (I think) and I have talked about modifying things so that you
can refer to a type by name before defining it as long as it is defined by
the end of the schema.

And I'm sure that if I wanted to trawl through JIRA, the lists and various
IRC discussions, we can find other instances.

That leaves aside things like GenAvro, which people have talked about
extending but isn't part of the base Avro spec.

Following along with these things requires exposing yourself to the full
flow of changes (from JIRA) and information (mailing list, IRC) and it isn't
terribly easy to keep track of what's being discussed, where, and how to
give comments.

Whatever the result of this thread is, AEPs or not, it would be nice to have
some way to know what changes are being proposed / discussed / implemented.

 - Bruce

On Tue, Jun 8, 2010 at 2:51 AM, Matt Massie <[EMAIL PROTECTED]> wrote:

> On Mon, Jun 7, 2010 at 9:43 AM, Doug Cutting <[EMAIL PROTECTED]> wrote:
> > On 06/04/2010 06:29 PM, Matt Massie wrote:
> >
> >> I would like to see us be a little more of a think-do-acrocy when it
> comes
> >> to important, high-level components of Avro (e.g. light-weight RPC).  It
> >> would be nice to have some simple consensus before code starts getting
> >> slung
> >> around;
> >>
> >
> > I'd rather we have proofs of concept before we standardize.  I often find
> > design issues while implementing.  I like to see two working
> implementations
> > before we promote something as an interoperability standard.  Little else
> of
> > substance has gone into the spec until it's been implemented twice.
> Of course we should have proofs of concept and I agree that you can find
> design issues while implementing ideas.  You can also find design issues by
> having more than one person review the design before it's implemented.
> I'm happy to drop my support for AEPs.  The main reason I supported them in
> the first place is that I felt they would be a good way of fostering
> communication.  If the team finds no value in them, then we should drop the
> idea for now.
> > otherwise, we'll see things in our spec like "The double is
> >> converted into a 64-bit integer using a method equivalent to Java's
> >> doubleToLongBits and then encoded in little-endian format."  Not to say
> we
> >> would ever do such a thing.  :)
> >>
> >
> > I don't see a fundamental problem with that.  It's a link to clear,
> > detailed documentation that describes a standard for serializing
> infinities
> > and NaNs, using zeroed mantissas.  If you'd rather make the spec more
> > standalone, then feel free to update that passage with these details.
>  But
> > there's nothing Java-specific in it.  This is similar to the link to
> > Protocol Buffers to define zig-zag encoding.  It's perhaps lazy writing,
> but
> > I don't think it's indicative of a technically flawed specification.  Do
> > you?
> >
> I do not think it indicates a technically flawed specification.
> -Matt
Bruce Mitchener 2010-06-11, 05:20
Jeff Hammerbacher 2010-06-11, 06:18
Scott Carey 2010-06-04, 00:00