Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro, mail # user - Defining optional records? (The usual way with unions doesn't work)


+
Tayssir John Gabbour 2012-09-12, 11:52
+
François Kawala 2012-09-12, 12:05
Copy link to this message
-
Re: Defining optional records? (The usual way with unions doesn't work)
Doug Cutting 2012-09-12, 16:57
On Wed, Sep 12, 2012 at 4:52 AM, Tayssir John Gabbour <[EMAIL PROTECTED]> wrote:
> SCHEMA:
> { "type": "record",
>   "name": "User",
>   "fields" : [
>     {"name": "username", "type": "string"},
>     {"name": "x", "type": ["null",
>                            {"type": "record",
>                             "name": "Test",
>                             "fields" : [
>                               {"name": "a", "type": "int"}
>                             ]}
>                           ]}
>   ]}
>
> DATA:
> {"username": "john", "x": {"a": 1}}
> {"username": "ryan", "x": {"a": 1}}

Non-null union values in Json need to be tagged with the intended branch.

http://avro.apache.org/docs/current/spec.html#json_encoding

So I think this needs to instead be something like:

{"username": "john", "x": {"Test": {"a": 1}}}

If you're generating this Json from Avro then you should use
JsonEncoder or the "tojson" command to do so, not the record's
.toString() method which doesn't tag unions correctly.  (Perhaps we
should fix the .toString() method to also tag unions, but that would
be an incompatible change.)

This has been asked before.  Perhaps we should add it to the FAQ?

Doug