Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro, mail # user - Do the values in the json object have to be ordered?


Copy link to this message
-
Do the values in the json object have to be ordered?
Jonathan Coveney 2013-04-04, 16:21
I think an example is most useful:

https://gist.github.com/jcoveney/5311795

I realize that the python implementation isn't as strict as the Java
implementation, though this result is a bit surprising.

Basically, is it the case that the Java generic writer expects that the
Json object's keys will be in the same order as the fields? This is what
the gist is trying to show. I have a simple record definition, and then two
identical json objects that match that definition, except for the order.

In python this works, which you'd expect, but in Java it does not. I get
the following:

First successful!
Exception in thread "main" java.lang.RuntimeException:
org.apache.avro.AvroTypeException: Expected field name first got second
    at com.spotify.hadoop.mapred.Hrm.main(Hrm.java:43)
Caused by: org.apache.avro.AvroTypeException: Expected field name first got
second
    at org.apache.avro.io.JsonDecoder.doAction(JsonDecoder.java:437)
    at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
    at org.apache.avro.io.JsonDecoder.advance(JsonDecoder.java:121)
    at org.apache.avro.io.JsonDecoder.readInt(JsonDecoder.java:148)
    at
org.apache.avro.io.ValidatingDecoder.readInt(ValidatingDecoder.java:83)
    at
org.apache.avro.generic.GenericDatumReader.readInt(GenericDatumReader.java:341)
    at
org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:146)
    at
org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:166)
    at
org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:138)
    at
org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:129)
    at com.spotify.hadoop.mapred.Hrm.main(Hrm.java:38)

Am I doing something dumb wrong? Per the JSON spec, objects are unordered
so it seems very problematic that it is expecting it to be ordered.

Thank you,
Jon