Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> Are optional fields possible?


+
Tim Robertson 2010-08-03, 10:07
Copy link to this message
-
Re: Are optional fields possible?
AFAIK there's no 'optional' kind of specifier. And 'defaults' are used
in readers only, not writers.

For storing 'null' in a field, I usually use the union solution and
wrap the field's schema as -- "type": ["string", null] --. (Note: This
comes with an additional Union-indicating byte cost.) I don't think
this works out for enumerations, however; so using a NULL symbol may
be the solution in that case?

On Tue, Aug 3, 2010 at 3:37 PM, Tim Robertson <[EMAIL PROTECTED]> wrote:
> Hi all,
>
> Is it possible in the schema to indicate that a filed is optional?
>
> I have things like:
>
>          {"name": "userId", "type": "string", "default": null},
>          {
>                "name": "level",
>                "type":
>                {
>                        "name": "LEVEL_TYPES",
>                        "type": "enum",
>                        "symbols" : ["DEBUG", "INFO", "WARN", "ERROR"]
>                },
>                "default": 0
>          },
>
> and Java serliazation code:
>
>                HttpTransceiver client = new HttpTransceiver(server);
>                LogEvent proxy = (LogEvent)
> SpecificRequestor.getClient(LogEvent.class, client);
>                Event e = new Event();
>                e.level=LEVEL_TYPES.DEBUG;
>                ... more setters
>                proxy.log(e);
>
> but I seem to have to set a value to every field in the schema, or else I get:
>
> Exception in thread "main" java.lang.NullPointerException
>        at org.apache.avro.specific.SpecificDatumWriter.writeEnum(SpecificDatumWriter.java:54)
>        at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:63)
>        at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:89)
>        at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:62)
>        at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:55)
>        at org.apache.avro.specific.SpecificRequestor.writeRequest(SpecificRequestor.java:69)
>        at org.apache.avro.ipc.Requestor.request(Requestor.java:109)
>        at org.apache.avro.specific.SpecificRequestor.invoke(SpecificRequestor.java:52)
>        at $Proxy0.log(Unknown Source)
>        at org.gbif.portalanalytics.client.GBIFLogger.info(GBIFLogger.java:41)
>        at org.gbif.portalanalytics.client.GBIFLogger.main(GBIFLogger.java:67)
>
> Is it expected to have to have a value on each field?
> For Enums this means I would need a NULL in the Enum explicitly I think.
>
> Thanks for any pointers!
> Tim
>

--
Harsh J
www.harshj.com
+
Tim Robertson 2010-08-03, 11:47