Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> How best to represent this in a union.


Copy link to this message
-
How best to represent this in a union.
Just trying to validate this is a good approach.We currently have a union
schema that we use to define a simple data item of type "any" (like the old
COM Variant):

{"name": "item", "type": [ "double",  "float",  "int",  "long",  "string",
 "DateTime",  "null"] }

I'd like to add another union member that works like "null", in that no
data ever gets written except the union discriminator itself. We are
streaming time-series data, and this value would have a special meaning of
"value is unchanged". I could make it an enum with just one value, but then
you would write at least two bytes. So I'm thinking I can make a record:

{
"type": "record",
"name": "Unchanged",
"fields":
[
{ "name": "item", "type": "null" }
]
}

and then my union becomes

{
 "name": "item", "type": [ "double",  "float",  "int",  "long",  "string",
 "DateTime",  "null", Unchanged]
}

Seem reasonable?