Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> Specific compiler task, separate schemas


Copy link to this message
-
Specific compiler task, separate schemas
Hello,

I'm running into what looks like a limitation of the ant task for compiling
specific classes from schemas.   If this is documented somewhere, I
apologize, I couldn't find it.

I have three schemas in separate files, where two are referenced by the
third:

{
    "type": "record",
    "name": "MemberInfo",
    "fields": [
        {"name": "name", "type": "FullName"},
        {"name": "address", "type": "Address"},
    ]
}

{
    "type": "record",
    "name": "FullName",
    "fields": [
        {"name": "first", "type": "string"},
        {"name": "middle", "type": "string", "default": ""},
        {"name": "last", "type": "string"}
    ]
}

{
    "type": "record",
    "name": "Address",
    "fields": [
        {"name": "street", "type": "string"},
        {"name": "city", "type": "string"},
        {"name": "state", "type": "string"},
        {"name": "zip", "type": "int"}
    ]
}

When using the ant schema task, if I put these all in one file, only the
first is compiled.  If I supply three separate files, a separate Parser is
created for each one, so the reference from MemberInfo to Address/FullName
can't be resolved.

Of course it works to put the embedded schemas inline in a single
MemberInfo schema, but that's not the way we'd like to do it.

Rather than using the ant task I am going to try writing a small utility to
call this method in SpecificCompiler:

  public static void compileSchema(File[] srcFiles, File dest) throws
IOException

This method uses a single Parser instance for all input files, so it should
work.  Of course, I'll need to supply the files in the right order
(MemberInfo last).

Anything I'm missing?

Thanks,
--mark
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB