Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> Specific compiler task, separate schemas

Copy link to this message
Specific compiler task, separate schemas

I'm running into what looks like a limitation of the ant task for compiling
specific classes from schemas.   If this is documented somewhere, I
apologize, I couldn't find it.

I have three schemas in separate files, where two are referenced by the

    "type": "record",
    "name": "MemberInfo",
    "fields": [
        {"name": "name", "type": "FullName"},
        {"name": "address", "type": "Address"},

    "type": "record",
    "name": "FullName",
    "fields": [
        {"name": "first", "type": "string"},
        {"name": "middle", "type": "string", "default": ""},
        {"name": "last", "type": "string"}

    "type": "record",
    "name": "Address",
    "fields": [
        {"name": "street", "type": "string"},
        {"name": "city", "type": "string"},
        {"name": "state", "type": "string"},
        {"name": "zip", "type": "int"}

When using the ant schema task, if I put these all in one file, only the
first is compiled.  If I supply three separate files, a separate Parser is
created for each one, so the reference from MemberInfo to Address/FullName
can't be resolved.

Of course it works to put the embedded schemas inline in a single
MemberInfo schema, but that's not the way we'd like to do it.

Rather than using the ant task I am going to try writing a small utility to
call this method in SpecificCompiler:

  public static void compileSchema(File[] srcFiles, File dest) throws

This method uses a single Parser instance for all input files, so it should
work.  Of course, I'll need to supply the files in the right order
(MemberInfo last).

Anything I'm missing?

Mark Hayes 2012-05-31, 01:04
Doug Cutting 2012-06-01, 00:05
Mark Hayes 2012-06-01, 18:55