Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> Specific compiler task, separate schemas


Copy link to this message
-
Specific compiler task, separate schemas
Hello,

I'm running into what looks like a limitation of the ant task for compiling
specific classes from schemas.   If this is documented somewhere, I
apologize, I couldn't find it.

I have three schemas in separate files, where two are referenced by the
third:

{
    "type": "record",
    "name": "MemberInfo",
    "fields": [
        {"name": "name", "type": "FullName"},
        {"name": "address", "type": "Address"},
    ]
}

{
    "type": "record",
    "name": "FullName",
    "fields": [
        {"name": "first", "type": "string"},
        {"name": "middle", "type": "string", "default": ""},
        {"name": "last", "type": "string"}
    ]
}

{
    "type": "record",
    "name": "Address",
    "fields": [
        {"name": "street", "type": "string"},
        {"name": "city", "type": "string"},
        {"name": "state", "type": "string"},
        {"name": "zip", "type": "int"}
    ]
}

When using the ant schema task, if I put these all in one file, only the
first is compiled.  If I supply three separate files, a separate Parser is
created for each one, so the reference from MemberInfo to Address/FullName
can't be resolved.

Of course it works to put the embedded schemas inline in a single
MemberInfo schema, but that's not the way we'd like to do it.

Rather than using the ant task I am going to try writing a small utility to
call this method in SpecificCompiler:

  public static void compileSchema(File[] srcFiles, File dest) throws
IOException

This method uses a single Parser instance for all input files, so it should
work.  Of course, I'll need to supply the files in the right order
(MemberInfo last).

Anything I'm missing?

Thanks,
--mark