Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> Do the values in the json object have to be ordered?


+
Jonathan Coveney 2013-04-04, 16:21
Copy link to this message
-
Re: Do the values in the json object have to be ordered?
On Thu, Apr 4, 2013 at 6:21 PM, Jonathan Coveney <[EMAIL PROTECTED]> wrote:
> I think an example is most useful:
>
> https://gist.github.com/jcoveney/5311795
>
> I realize that the python implementation isn't as strict as the Java
> implementation, though this result is a bit surprising.
>
> Basically, is it the case that the Java generic writer expects that the Json
> object's keys will be in the same order as the fields? This is what the gist
> is trying to show. I have a simple record definition, and then two identical
> json objects that match that definition, except for the order.
>
> In python this works, which you'd expect, but in Java it does not. I get the
> following:
>
> First successful!
> Exception in thread "main" java.lang.RuntimeException:
> org.apache.avro.AvroTypeException: Expected field name first got second
>     at com.spotify.hadoop.mapred.Hrm.main(Hrm.java:43)
> Caused by: org.apache.avro.AvroTypeException: Expected field name first got
> second
>     at org.apache.avro.io.JsonDecoder.doAction(JsonDecoder.java:437)
>     at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
>     at org.apache.avro.io.JsonDecoder.advance(JsonDecoder.java:121)
>     at org.apache.avro.io.JsonDecoder.readInt(JsonDecoder.java:148)
>     at
> org.apache.avro.io.ValidatingDecoder.readInt(ValidatingDecoder.java:83)
>     at
> org.apache.avro.generic.GenericDatumReader.readInt(GenericDatumReader.java:341)
>     at
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:146)
>     at
> org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:166)
>     at
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:138)
>     at
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:129)
>     at com.spotify.hadoop.mapred.Hrm.main(Hrm.java:38)
>
> Am I doing something dumb wrong? Per the JSON spec, objects are unordered so
> it seems very problematic that it is expecting it to be ordered.
>
> Thank you,
> Jon

Indeed, this contradicts the JSON spec. Order does not matter in JSON.

Jackson however deserializes JSON with a LinkedHashMap by default. I
suppose Avro takes advantage of this, but it still contradicts the
spec.

--
Francis Galiegue, [EMAIL PROTECTED]
JSON Schema in Java: http://json-schema-validator.herokuapp.com
+
Jonathan Coveney 2013-04-04, 17:28
+
Philip Zeyliger 2013-04-04, 19:47
+
Jonathan Coveney 2013-04-04, 20:10
+
Jonathan Coveney 2013-04-05, 08:22
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB