Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> Question about Avro "records"


+
Francis Galiegue 2013-02-27, 21:47
+
Pankaj Shroff 2013-02-27, 21:54
+
Francis Galiegue 2013-02-27, 22:14
+
Pankaj Shroff 2013-02-27, 22:21
+
Francis Galiegue 2013-02-27, 22:52
+
Doug Cutting 2013-02-27, 22:20
+
Francis Galiegue 2013-02-27, 22:51
+
Doug Cutting 2013-02-27, 23:10
+
Francis Galiegue 2013-02-27, 23:55
+
Doug Cutting 2013-02-28, 01:12
+
Jeremy Kahn 2013-02-28, 01:30
+
Doug Cutting 2013-02-28, 17:11
Copy link to this message
-
Re: Question about Avro "records"
On Thu, Feb 28, 2013 at 2:30 AM, Jeremy Kahn <[EMAIL PROTECTED]> wrote:
> There seems to be no way to easily use the avro libraries in Python (where I
> feel most qualified to comment) to encode generics with "missing default
> values" and have them transmitted in well-formed avro binary.
>
> If you fill in the "missing" default values, the Python libraries will
> transmit correctly.
>
> I'd be happy to add methods to the avro.RecordSchema objects (in the Python
> libraries) that "fill defaults" on missing member fields of a record,
> recursively (which probably means method extension of other schema classes
> as well). Shall I open a JIRA ticket for this for 1.7.5?
>
> (Does providing this for Python pot me on the hook for such a thing in other
> implementation languages? I hope not.)
>
> For backwards compatibility (and probably to avoid unnecessary data
> traversal), you'll probably want to explicitly ask the schema to fill in
> defaults before transmission in the cases where you'd like to generate the
> impoverished JSON from your example.
>
> This seems related to earlier discussion today about designing constructors
> to generate defaults already filled in.
>

In fact, I was just asking how this should be handled because I have
just finished writing an Avro schema to JSON Schema conversion
processor (which I'll put online soon), so I wanted to be as accurate
as possible when generating schemas ;)

Right now the generated schemas require all properties, even ones
having default values. I was wondering if that was the right thing to
do...

(and next I'm attacking the reverse: JSON Schema to Avro schema...)

--
Francis Galiegue, [EMAIL PROTECTED]
JSON Schema in Java: http://json-schema-validator.herokuapp.com