Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro, mail # user - Best practice for versioning IDLs?

Copy link to this message
Re: Best practice for versioning IDLs?
George Fletcher 2011-11-29, 19:33
Hi Scott,

Thanks for the suggestion. I like the metadata approach!

I just tried that property on one of my protocol IDL files and it didn't
work (parsing error). I discussed this is Doug Cutting as part of JIRA
ticket 886 (https://issues.apache.org/jira/browse/AVRO-886). However,
that ticket is now closed so I should probably open a new one unless I'm
doing something wrong. I was hoping to work on this a while back but got
busy with other tasks. Looks like now might be the right time:)


On 11/29/11 1:22 PM, Scott Carey wrote:
> I don't think there are yet best practices for what you are trying to do.
> However, I suggest you first consider embedding the version as metadata in
> the schema, rather than data.  If you put it in a Record, it will be data
> serialized with every record.  If you put it as schema metadata, it will
> only exist in the schemas and not the data.
> In raw JSON schema form, the metadata can be added to any named type:
> Record, Fixed, Enum, Protocol.   The "doc" field is a special named
> metadata field, you can use it or add your own:
> {
>    "namespace": "com.acme",
>    "protocol": "HelloWorld",
>    "doc": "Protocol Greetings",
>    "acme.version": "1.22.3",
>    "types": [
>      {"name": "Greeting", "type": "record", "fields": [
>        {"name": "message", "type": "string"}]},
>      {"name": "Curse", "type": "error", "fields": [
>        {"name": "message", "type": "string"}]}
>    ],
>    "messages": {
>      "hello": {
>        "doc": "Say hello.",
>        "request": [{"name": "greeting", "type": "Greeting" }],
>        "response": "Greeting",
>        "errors": ["Curse"]
>      }
>    }
> }
> http://avro.apache.org/docs/current/spec.html#Protocol+Declaration
> For IDL, it should be possible to add a property using the
> @propname("propval") annotation on the protocol.
> http://avro.apache.org/docs/current/idl.html#defining_protocol
> I have not tried this myself however.
> If I had the setup to test it now, I would try to see if the below AvroIDL
> creates an empty protocol with the "acme.version" property set:
> @acme.version("1.22.3")
> @namespace("com.acme")
> protocol HelloWorld {
> }
> On 11/29/11 9:20 AM, "George Fletcher"<[EMAIL PROTECTED]>  wrote:
>>     Hi,
>>       I'd like to incorporate a semver.org style versioning structure
>>       for the IDL files we are using. The IDLs represent interfaces of
>>       services (ala SOA).
>>       We currently manage our IDL files separately from the
>>       implementation as multiple services might use the same IDL. This
>>       makes it critical to have the IDL's understand their version. I'd
>>       like to see our build process be able to inject into the IDL the
>>       version from the build environment (currently maven). Another
>>       option would be to define the version within the IDL. However, the
>>       only way I can think of to do this, is to create a "Version"
>>       Record within each IDL and then maybe have the Record contains 3
>>       string fields (major, minor, patch).
>>       Just wondering if there are any best practices already established
>>       for this kind of IDL versioning.
>>       Thanks,
>>       George

Chief Architect                   AIM:  gffletch
Identity Services Engineering     Work: [EMAIL PROTECTED]
AOL Inc.                          Home: [EMAIL PROTECTED]
Mobile: +1-703-462-3494           Blog: http://practicalid.blogspot.com
Office: +1-703-265-2544           Twitter: http://twitter.com/gffletch