Hi all,

Is it possible in the schema to indicate that a filed is optional?

I have things like:

          {"name": "userId", "type": "string", "default": null},
          {
           "name": "level",
           "type":
           {
          "name": "LEVEL_TYPES",
           "type": "enum",
          "symbols" : ["DEBUG", "INFO", "WARN", "ERROR"]
       },
       "default": 0
          },

and Java serliazation code:

HttpTransceiver client = new HttpTransceiver(server);
LogEvent proxy = (LogEvent)
SpecificRequestor.getClient(LogEvent.class, client);
Event e = new Event();
e.level=LEVEL_TYPES.DEBUG;
... more setters
proxy.log(e);

but I seem to have to set a value to every field in the schema, or else I get:

Exception in thread "main" java.lang.NullPointerException
at org.apache.avro.specific.SpecificDatumWriter.writeEnum(SpecificDatumWriter.java:54)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:63)
at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:89)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:62)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:55)
at org.apache.avro.specific.SpecificRequestor.writeRequest(SpecificRequestor.java:69)
at org.apache.avro.ipc.Requestor.request(Requestor.java:109)
at org.apache.avro.specific.SpecificRequestor.invoke(SpecificRequestor.java:52)
at $Proxy0.log(Unknown Source)
at org.gbif.portalanalytics.client.GBIFLogger.info(GBIFLogger.java:41)
at org.gbif.portalanalytics.client.GBIFLogger.main(GBIFLogger.java:67)

Is it expected to have to have a value on each field?
For Enums this means I would need a NULL in the Enum explicitly I think.

Thanks for any pointers!
Tim

AFAIK there's no 'optional' kind of specifier. And 'defaults' are used
in readers only, not writers.

For storing 'null' in a field, I usually use the union solution and
wrap the field's schema as -- "type": ["string", null] --. (Note: This
comes with an additional Union-indicating byte cost.) I don't think
this works out for enumerations, however; so using a NULL symbol may
be the solution in that case?

On Tue, Aug 3, 2010 at 3:37 PM, Tim Robertson <[EMAIL PROTECTED]> wrote:
> Hi all,
>
> Is it possible in the schema to indicate that a filed is optional?
>
> I have things like:
>
>          {"name": "userId", "type": "string", "default": null},
>          {
>                "name": "level",
>                "type":
>                {
>                        "name": "LEVEL_TYPES",
>                        "type": "enum",
>                        "symbols" : ["DEBUG", "INFO", "WARN", "ERROR"]
>                },
>                "default": 0
>          },
>
> and Java serliazation code:
>
>                HttpTransceiver client = new HttpTransceiver(server);
>                LogEvent proxy = (LogEvent)
> SpecificRequestor.getClient(LogEvent.class, client);
>                Event e = new Event();
>                e.level=LEVEL_TYPES.DEBUG;
>                ... more setters
>                proxy.log(e);
>
> but I seem to have to set a value to every field in the schema, or else I get:
>
> Exception in thread "main" java.lang.NullPointerException
>        at org.apache.avro.specific.SpecificDatumWriter.writeEnum(SpecificDatumWriter.java:54)
>        at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:63)
>        at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:89)
>        at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:62)
>        at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:55)
>        at org.apache.avro.specific.SpecificRequestor.writeRequest(SpecificRequestor.java:69)
>        at org.apache.avro.ipc.Requestor.request(Requestor.java:109)
>        at org.apache.avro.specific.SpecificRequestor.invoke(SpecificRequestor.java:52)
>        at $Proxy0.log(Unknown Source)
>        at org.gbif.portalanalytics.client.GBIFLogger.info(GBIFLogger.java:41)
>        at org.gbif.portalanalytics.client.GBIFLogger.main(GBIFLogger.java:67)
>
> Is it expected to have to have a value on each field?
> For Enums this means I would need a NULL in the Enum explicitly I think.
>
> Thanks for any pointers!
> Tim
>

--
Harsh J
www.harshj.com

Thanks - I had glanced over the union bit in the spec, but that is
what I was looking for!

Cheers,
Tim

On Tue, Aug 3, 2010 at 1:40 PM, Harsh J <[EMAIL PROTECTED]> wrote:
> AFAIK there's no 'optional' kind of specifier. And 'defaults' are used
> in readers only, not writers.
>
> For storing 'null' in a field, I usually use the union solution and
> wrap the field's schema as -- "type": ["string", null] --. (Note: This
> comes with an additional Union-indicating byte cost.) I don't think
> this works out for enumerations, however; so using a NULL symbol may
> be the solution in that case?
>
> On Tue, Aug 3, 2010 at 3:37 PM, Tim Robertson <[EMAIL PROTECTED]> wrote:
>> Hi all,
>>
>> Is it possible in the schema to indicate that a filed is optional?
>>
>> I have things like:
>>
>>          {"name": "userId", "type": "string", "default": null},
>>          {
>>                "name": "level",
>>                "type":
>>                {
>>                        "name": "LEVEL_TYPES",
>>                        "type": "enum",
>>                        "symbols" : ["DEBUG", "INFO", "WARN", "ERROR"]
>>                },
>>                "default": 0
>>          },
>>
>> and Java serliazation code:
>>
>>                HttpTransceiver client = new HttpTransceiver(server);
>>                LogEvent proxy = (LogEvent)
>> SpecificRequestor.getClient(LogEvent.class, client);
>>                Event e = new Event();
>>                e.level=LEVEL_TYPES.DEBUG;
>>                ... more setters
>>                proxy.log(e);
>>
>> but I seem to have to set a value to every field in the schema, or else I get:
>>
>> Exception in thread "main" java.lang.NullPointerException
>>        at org.apache.avro.specific.SpecificDatumWriter.writeEnum(SpecificDatumWriter.java:54)
>>        at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:63)
>>        at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:89)
>>        at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:62)
>>        at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:55)
>>        at org.apache.avro.specific.SpecificRequestor.writeRequest(SpecificRequestor.java:69)
>>        at org.apache.avro.ipc.Requestor.request(Requestor.java:109)
>>        at org.apache.avro.specific.SpecificRequestor.invoke(SpecificRequestor.java:52)
>>        at $Proxy0.log(Unknown Source)
>>        at org.gbif.portalanalytics.client.GBIFLogger.info(GBIFLogger.java:41)
>>        at org.gbif.portalanalytics.client.GBIFLogger.main(GBIFLogger.java:67)
>>
>> Is it expected to have to have a value on each field?
>> For Enums this means I would need a NULL in the Enum explicitly I think.
>>
>> Thanks for any pointers!
>> Tim
>>
>
>
>
> --
> Harsh J
> www.harshj.com
>