Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro, mail # user - Do the values in the json object have to be ordered?


Copy link to this message
-
Re: Do the values in the json object have to be ordered?
Francis Galiegue 2013-04-04, 16:46
On Thu, Apr 4, 2013 at 6:21 PM, Jonathan Coveney <[EMAIL PROTECTED]> wrote:
> I think an example is most useful:
>
> https://gist.github.com/jcoveney/5311795
>
> I realize that the python implementation isn't as strict as the Java
> implementation, though this result is a bit surprising.
>
> Basically, is it the case that the Java generic writer expects that the Json
> object's keys will be in the same order as the fields? This is what the gist
> is trying to show. I have a simple record definition, and then two identical
> json objects that match that definition, except for the order.
>
> In python this works, which you'd expect, but in Java it does not. I get the
> following:
>
> First successful!
> Exception in thread "main" java.lang.RuntimeException:
> org.apache.avro.AvroTypeException: Expected field name first got second
>     at com.spotify.hadoop.mapred.Hrm.main(Hrm.java:43)
> Caused by: org.apache.avro.AvroTypeException: Expected field name first got
> second
>     at org.apache.avro.io.JsonDecoder.doAction(JsonDecoder.java:437)
>     at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
>     at org.apache.avro.io.JsonDecoder.advance(JsonDecoder.java:121)
>     at org.apache.avro.io.JsonDecoder.readInt(JsonDecoder.java:148)
>     at
> org.apache.avro.io.ValidatingDecoder.readInt(ValidatingDecoder.java:83)
>     at
> org.apache.avro.generic.GenericDatumReader.readInt(GenericDatumReader.java:341)
>     at
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:146)
>     at
> org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:166)
>     at
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:138)
>     at
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:129)
>     at com.spotify.hadoop.mapred.Hrm.main(Hrm.java:38)
>
> Am I doing something dumb wrong? Per the JSON spec, objects are unordered so
> it seems very problematic that it is expecting it to be ordered.
>
> Thank you,
> Jon

Indeed, this contradicts the JSON spec. Order does not matter in JSON.

Jackson however deserializes JSON with a LinkedHashMap by default. I
suppose Avro takes advantage of this, but it still contradicts the
spec.

--
Francis Galiegue, [EMAIL PROTECTED]
JSON Schema in Java: http://json-schema-validator.herokuapp.com