Avro users,

I have an avpr file and I'd like to include other avpr files so that I can
split my schemas into files with one file per namespace. Is this possible
or do I need to use Avro IDL if I want to use imports.

Thanks
Jeremy

On 02/16/2012 07:06 AM, Jeremy Lewi wrote:
> I have an avpr file and I'd like to include other avpr files so that I
> can split my schemas into files with one file per namespace. Is this
> possible or do I need to use Avro IDL if I want to use imports.

You could use the same API that the IDL parser uses.  It copies each of
the types and messages from included protocols to a new protocol
instance (http://s.apache.org/wj).

Perhaps we should add a Protocol.Parser API like Schema.Parser.  One
could then repeatedly call parser.parseProtocol(avprFile) then call
parser.getProtocol() to return a protocol with all of the types and
methods.  Types defined in earlier files would be available for
reference by subsequent files.  This would make things somewhat simpler
than the current API does.  If that sounds like it would be useful,
please file an issue in Jira.

Doug

I'm not using a parser. I'm just using the avro compiler to generate source
files which I use in my code (I'm running mapreduce jobs whose
inputs/outputs are avro files).
J

On Thu, Feb 16, 2012 at 8:18 AM, Doug Cutting <[EMAIL PROTECTED]> wrote:

> On 02/16/2012 07:06 AM, Jeremy Lewi wrote:
> > I have an avpr file and I'd like to include other avpr files so that I
> > can split my schemas into files with one file per namespace. Is this
> > possible or do I need to use Avro IDL if I want to use imports.
>
> You could use the same API that the IDL parser uses.  It copies each of
> the types and messages from included protocols to a new protocol
> instance (http://s.apache.org/wj).
>
> Perhaps we should add a Protocol.Parser API like Schema.Parser.  One
> could then repeatedly call parser.parseProtocol(avprFile) then call
> parser.getProtocol() to return a protocol with all of the types and
> methods.  Types defined in earlier files would be available for
> reference by subsequent files.  This would make things somewhat simpler
> than the current API does.  If that sounds like it would be useful,
> please file an issue in Jira.
>
> Doug
>

On 02/16/2012 06:41 PM, Jeremy Lewi wrote:
> I'm not using a parser. I'm just using the avro compiler to generate
> source files which I use in my code (I'm running mapreduce jobs whose
> inputs/outputs are avro files).

I think this is possible since AVRO-877 (in release 1.5.3).  One can
specify multiple protocols on the compiler command line and types from
prior protocols are available in subsequent protocols.  For example:

java -jar avro-tools.jar compile protocol foo.avpr bar.avpr target/java

Would that work?

Doug

Thanks for the tip. I'll play around with that when I have more time. Right
now I think I'll stick with one monolithic avpr file.

J

On Fri, Feb 17, 2012 at 9:54 AM, Doug Cutting <[EMAIL PROTECTED]> wrote:

> On 02/16/2012 06:41 PM, Jeremy Lewi wrote:
> > I'm not using a parser. I'm just using the avro compiler to generate
> > source files which I use in my code (I'm running mapreduce jobs whose
> > inputs/outputs are avro files).
>
> I think this is possible since AVRO-877 (in release 1.5.3).  One can
> specify multiple protocols on the compiler command line and types from
> prior protocols are available in subsequent protocols.  For example:
>
> java -jar avro-tools.jar compile protocol foo.avpr bar.avpr target/java
>
> Would that work?
>
> Doug
>