Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> Are optional fields possible?


+
Tim Robertson 2010-08-03, 10:07
Copy link to this message
-
Re: Are optional fields possible?
AFAIK there's no 'optional' kind of specifier. And 'defaults' are used
in readers only, not writers.

For storing 'null' in a field, I usually use the union solution and
wrap the field's schema as -- "type": ["string", null] --. (Note: This
comes with an additional Union-indicating byte cost.) I don't think
this works out for enumerations, however; so using a NULL symbol may
be the solution in that case?

On Tue, Aug 3, 2010 at 3:37 PM, Tim Robertson <[EMAIL PROTECTED]> wrote:
> Hi all,
>
> Is it possible in the schema to indicate that a filed is optional?
>
> I have things like:
>
>          {"name": "userId", "type": "string", "default": null},
>          {
>                "name": "level",
>                "type":
>                {
>                        "name": "LEVEL_TYPES",
>                        "type": "enum",
>                        "symbols" : ["DEBUG", "INFO", "WARN", "ERROR"]
>                },
>                "default": 0
>          },
>
> and Java serliazation code:
>
>                HttpTransceiver client = new HttpTransceiver(server);
>                LogEvent proxy = (LogEvent)
> SpecificRequestor.getClient(LogEvent.class, client);
>                Event e = new Event();
>                e.level=LEVEL_TYPES.DEBUG;
>                ... more setters
>                proxy.log(e);
>
> but I seem to have to set a value to every field in the schema, or else I get:
>
> Exception in thread "main" java.lang.NullPointerException
>        at org.apache.avro.specific.SpecificDatumWriter.writeEnum(SpecificDatumWriter.java:54)
>        at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:63)
>        at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:89)
>        at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:62)
>        at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:55)
>        at org.apache.avro.specific.SpecificRequestor.writeRequest(SpecificRequestor.java:69)
>        at org.apache.avro.ipc.Requestor.request(Requestor.java:109)
>        at org.apache.avro.specific.SpecificRequestor.invoke(SpecificRequestor.java:52)
>        at $Proxy0.log(Unknown Source)
>        at org.gbif.portalanalytics.client.GBIFLogger.info(GBIFLogger.java:41)
>        at org.gbif.portalanalytics.client.GBIFLogger.main(GBIFLogger.java:67)
>
> Is it expected to have to have a value on each field?
> For Enums this means I would need a NULL in the Enum explicitly I think.
>
> Thanks for any pointers!
> Tim
>

--
Harsh J
www.harshj.com
+
Tim Robertson 2010-08-03, 11:47
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB