Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> Defining optional records? (The usual way with unions doesn't work)


+
Tayssir John Gabbour 2012-09-12, 11:52
+
François Kawala 2012-09-12, 12:05
Copy link to this message
-
Re: Defining optional records? (The usual way with unions doesn't work)
On Wed, Sep 12, 2012 at 4:52 AM, Tayssir John Gabbour <[EMAIL PROTECTED]> wrote:
> SCHEMA:
> { "type": "record",
>   "name": "User",
>   "fields" : [
>     {"name": "username", "type": "string"},
>     {"name": "x", "type": ["null",
>                            {"type": "record",
>                             "name": "Test",
>                             "fields" : [
>                               {"name": "a", "type": "int"}
>                             ]}
>                           ]}
>   ]}
>
> DATA:
> {"username": "john", "x": {"a": 1}}
> {"username": "ryan", "x": {"a": 1}}

Non-null union values in Json need to be tagged with the intended branch.

http://avro.apache.org/docs/current/spec.html#json_encoding

So I think this needs to instead be something like:

{"username": "john", "x": {"Test": {"a": 1}}}

If you're generating this Json from Avro then you should use
JsonEncoder or the "tojson" command to do so, not the record's
.toString() method which doesn't tag unions correctly.  (Perhaps we
should fix the .toString() method to also tag unions, but that would
be an incompatible change.)

This has been asked before.  Perhaps we should add it to the FAQ?

Doug
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB