Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> Question about Avro "records"


Copy link to this message
-
Re: Question about Avro "records"
On Thu, Feb 28, 2013 at 2:30 AM, Jeremy Kahn <[EMAIL PROTECTED]> wrote:
> There seems to be no way to easily use the avro libraries in Python (where I
> feel most qualified to comment) to encode generics with "missing default
> values" and have them transmitted in well-formed avro binary.
>
> If you fill in the "missing" default values, the Python libraries will
> transmit correctly.
>
> I'd be happy to add methods to the avro.RecordSchema objects (in the Python
> libraries) that "fill defaults" on missing member fields of a record,
> recursively (which probably means method extension of other schema classes
> as well). Shall I open a JIRA ticket for this for 1.7.5?
>
> (Does providing this for Python pot me on the hook for such a thing in other
> implementation languages? I hope not.)
>
> For backwards compatibility (and probably to avoid unnecessary data
> traversal), you'll probably want to explicitly ask the schema to fill in
> defaults before transmission in the cases where you'd like to generate the
> impoverished JSON from your example.
>
> This seems related to earlier discussion today about designing constructors
> to generate defaults already filled in.
>

In fact, I was just asking how this should be handled because I have
just finished writing an Avro schema to JSON Schema conversion
processor (which I'll put online soon), so I wanted to be as accurate
as possible when generating schemas ;)

Right now the generated schemas require all properties, even ones
having default values. I was wondering if that was the right thing to
do...

(and next I'm attacking the reverse: JSON Schema to Avro schema...)

--
Francis Galiegue, [EMAIL PROTECTED]
JSON Schema in Java: http://json-schema-validator.herokuapp.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB