Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> Using optional fields (nullable unions) with the C-API


+
Markus Resch 2011-11-21, 17:26
Copy link to this message
-
Re: Using optional fields (nullable unions) with the C-API
> I'm using a record like this
>
> {
>    "type" : "record",
>    "name" : "TestPacket",
>    "fields" : [
>        { "name" : "Id" , "type" : "int"},
>        { "name" : "Value" , "type" : ["string","null"]}
> ]
> }
>
> But if I just not write anything to the 'Value' filed,
> avro_writer_memory() refuses to validate it. How do you normally use the
> optional fields with the C-API?

Hi Markus

In Avro, an “optional” field is really just a common pattern where you've got a union, with a “null” for one branch, and some other type for the second branch.  And with all union types, you have to explicitly tell the Avro library which branch is active for a particular value.  (Think of the union as a container that must always contain exactly one value, but where the type of that value can change.)

So, if you're using the new value API, you'd use something like:

  avro_value_t  rec = /* given */;
  avro_value_t  field, branch;
  avro_value_get_by_name(rec, "Value", &field, NULL);
  avro_value_set_branch(&field, 1, &branch);
  avro_value_set_null(&branch);

and if you're using the old datum API:

  avro_datum_t  rec = /* given */;
  avro_datum_t  field, branch;
  avro_record_get(rec, "Value", &field);
  avro_union_set_discriminant(field, 1, &branch);
  /* no avro_null_set in the old API */

In set_branch/set_discriminant, the 1 refers to the index of the "null" type in the union.  I tend to use branch 0 for the null type in these kinds of unions, but that's just a matter of aesthetics.

cheers
–doug
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB