|
|
-
Exposing a constant in an Avro Schema
Andrew Kenworthy 2011-11-14, 16:03
Hi,
I would like to "embed" a schema version number in the schema that I use for writing data: it would be read-only so that I can determine later on which version of my avro schema was used. The best I could come up with is to (ab)use an enum with a single value like this, as I couldn't find any way to define a constant:
{"type":"enum","name":"version_1_1","doc":"enum indicating avro write schema version 1.1","symbols":["VERSION_1_1"]}
Is there a better way to register a constant value that has no meaning within the avro data file, other than to expose some kind of meta information?
Thanks,
Andrew Kenworthy
+
Andrew Kenworthy 2011-11-14, 16:03
-
Re: Exposing a constant in an Avro Schema
Scott Carey 2011-11-14, 20:09
Named types (records, fields, fixed, enum) can store arbitrary user properties attached to the schema ( similar to "doc" but no special meaning).
Do you want this constant to be in every instance of your data object? If so, the enum is one way to do it. If you simply want to push metadata along with the schema, use the schema properties, they are name-value pairs. For example you can have "myVersion" attached to your schema for a record:
{"type":"record", "name":"bar.baz.FooRecord", "myVersion":"1.1", "fields": { {"name":"field1", "type":"int"}, } }
On 11/14/11 8:03 AM, "Andrew Kenworthy" <[EMAIL PROTECTED]> wrote:
> Hi, > > I would like to "embed" a schema version number in the schema that I use for > writing data: it would be read-only so that I can determine later on which > version of my avro schema was used. The best I could come up with is to > (ab)use an enum with a single value like this, as I couldn't find any way to > define a constant: > > {"type":"enum","name":"version_1_1","doc":"enum indicating avro write schema > version 1.1","symbols":["VERSION_1_1"]} > > Is there a better way to register a constant value that has no meaning within > the avro data file, other than to expose some kind of meta information? > > Thanks, > > Andrew Kenworthy >
+
Scott Carey 2011-11-14, 20:09
-
Re: Exposing a constant in an Avro Schema
Andrew Kenworthy 2011-11-15, 09:14
Hi Scott,
it's the latter I need; simply the ability to pass meta-data with my schema, so the user property is just what I need.
Thanks for your help!
Andrew
>________________________________ >From: Scott Carey <[EMAIL PROTECTED]> >To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>; Andrew Kenworthy <[EMAIL PROTECTED]> >Sent: Monday, November 14, 2011 9:09 PM >Subject: Re: Exposing a constant in an Avro Schema > > >Named types (records, fields, fixed, enum) can store arbitrary user properties attached to the schema ( similar to "doc" but no special meaning). > > >Do you want this constant to be in every instance of your data object? If so, the enum is one way to do it. >If you simply want to push metadata along with the schema, use the schema properties, they are name-value pairs. For example you can have "myVersion" attached to your schema for a record: > > >{"type":"record", "name":"bar.baz.FooRecord", "myVersion":"1.1", "fields": { > {"name":"field1", "type":"int"}, > … > } >} > >On 11/14/11 8:03 AM, "Andrew Kenworthy" <[EMAIL PROTECTED]> wrote: > > >Hi, >> >> >>I would like to "embed" a schema version number in the schema that I use for writing data: it would be read-only so that I can determine later on which version of my avro schema was used. The best I could come up with is to (ab)use an enum with a single value like this, as I couldn't find any way to define a constant: >> >> >>{"type":"enum","name":"version_1_1","doc":"enum indicating avro write schema version 1.1","symbols":["VERSION_1_1"]} >> >> >>Is there a better way to register a constant value that has no meaning within the avro data file, other than to expose some kind of meta information? >> >> >>Thanks, >> >> >>Andrew Kenworthy >> >> > >
+
Andrew Kenworthy 2011-11-15, 09:14
|
|