Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro, mail # user - Issue writing union in avro?


Copy link to this message
-
Re: Issue writing union in avro?
Jonathan Coveney 2013-04-05, 11:59
Ok, I figured out the issue:

If you make string c the following:
String c = "{\"name\": \"Alyssa\", \"favorite_number\": {\"int\": 256},
\"favorite_color\": {\"string\": \"blue\"}}";

Then this works.

This represents a divergence between the python and the Java
implementation... the above does not work in Python, but it does work in
Java. And of course, vice versa.

I think I know how to fix this (and can file a bug with my reproduction and
the fix), but I'm not sure which one is the expected case? Which
implementation is wrong?

Thanks
2013/4/5 Jonathan Coveney <[EMAIL PROTECTED]>

> Correction: the issue is when reading the string according to the avro
> schema, not on writing. it fails before I get a chance to write :)
>
>
> 2013/4/5 Jonathan Coveney <[EMAIL PROTECTED]>
>
>> I implemented essentially the Java avro example but using the
>> GenericDatumWriter and GenericDatumReader and hit an issue.
>>
>> https://gist.github.com/jcoveney/5317904
>>
>> This is the error:
>> Exception in thread "main" java.lang.RuntimeException:
>> org.apache.avro.AvroTypeException: Expected start-union. Got
>> VALUE_NUMBER_INT
>>     at com.spotify.hadoop.mapred.Hrm.main(Hrm.java:45)
>> Caused by: org.apache.avro.AvroTypeException: Expected start-union. Got
>> VALUE_NUMBER_INT
>>     at org.apache.avro.io.JsonDecoder.error(JsonDecoder.java:697)
>>     at org.apache.avro.io.JsonDecoder.readIndex(JsonDecoder.java:441)
>>     at
>> org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:229)
>>     at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
>>     at
>> org.apache.avro.io.ResolvingDecoder.readIndex(ResolvingDecoder.java:206)
>>     at
>> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:152)
>>     at
>> org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:177)
>>     at
>> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:148)
>>     at
>> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:139)
>>     at com.spotify.hadoop.mapred.Hrm.main(Hrm.java:38)
>>
>> Am I doing something wrong? Is this a bug? I'm digging in now but am
>> curious if anyone has seen this before?
>>
>> I get the feeling I am working with Avro in a way that most people do not
>> :)
>>
>>
>