Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> Picking up default value for a union?


+
Jonathan Coveney 2013-04-09, 09:06
+
Jonathan Coveney 2013-04-09, 09:31
Copy link to this message
-
Re: Picking up default value for a union?
Stepping through the code, it looks like the code only uses defaults for
writing, not for reading. IE at read time it assumes that the defaults were
already filled in. It seems like if the reader evolved the schema to
include new fields, it would be desirable for the defaults to get filled in
if not present? But stepping through, on reading the defaults are
completely ignored.
2013/4/9 Jonathan Coveney <[EMAIL PROTECTED]>

> Please note: {"name":"hey", "type":"record",
> "fields":[{"name":"a","type":["null","string"],"default":"null"}]} also
> doesn't work
>
>
> 2013/4/9 Jonathan Coveney <[EMAIL PROTECTED]>
>
>> I have the following schema: {"name":"hey", "type":"record",
>> "fields":[{"name":"a","type":["null","string"],"default":null}]}
>>
>> I am trying to deserialize the following against this schema using Java
>> and the GenericDatumReader: {}
>>
>> I get the following error:
>> Caused by: org.apache.avro.AvroTypeException: Expected start-union. Got
>> END_OBJECT
>>     at org.apache.avro.io.JsonDecoder.error(JsonDecoder.java:697)
>>     at org.apache.avro.io.JsonDecoder.readIndex(JsonDecoder.java:441)
>>     at
>> org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:229)
>>     at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
>>     at
>> org.apache.avro.io.ResolvingDecoder.readIndex(ResolvingDecoder.java:206)
>>     at
>> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:152)
>>     at
>> org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:177)
>>     at
>> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:148)
>>     at
>> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:139)
>>     at com.spotify.hadoop.JsonTester.main(JsonTester.java:40)
>>
>> I'm not seeing any immediate issues online around this...is this
>> expected? I'm reading it in as such:
>>
>> Schema avroSchema = new Schema.Parser().parse(schemaLine);
>> GenericDatumReader<Object> reader = new
>> GenericDatumReader<Object>(avroSchema);
>> Object datum = reader.read(null,
>> DecoderFactory.get().jsonDecoder(avroSchema, dataLine));
>>
>> I'm going to see what's up and why it isn't picking up the default, but
>> imagined you guys might know what's up?
>>
>> Thanks,
>> Jon
>>
>
>
+
Martin Kleppmann 2013-04-10, 03:42
+
Scott Carey 2013-04-11, 04:21
+
Jonathan Coveney 2013-04-11, 22:22