+
Steven Willis 2012-12-11, 19:28
+
Dave McAlpin 2012-12-11, 20:31
-
RE: How to handle schema dependenciesSteven Willis 2012-12-11, 21:42
Thanks Dave,
I was thinking about doing something like that (adding the schemas as resources in the jars). It just seems like a lot of work for something that could be automatic. It would be nice if during schema parsing we could specify a classpath to be used for dynamic lookup of external schemas. That way Schema.parse could look up already schemas that have already been compiled like this:
package com.compete.util;
import org.apache.avro.Schema;
import org.apache.avro.generic.GenericContainer;
public class Schemas {
public static void main(String args[]) {
String name = "Date";
String space = "com.compete.avro";
try {
// If we were inside Schema.parse we would probably do:
// Class.forName(new Name(name, space).toString())
Class<?> cls = Class.forName(space+"."+name);
GenericContainer record = (GenericContainer)cls.newInstance();
Schema schema = record.getSchema();
// if we were inside Schema.parse(JsonNode schema, Names names) we
// could now just call: names.put(new Name(name, space), schema);
System.out.println(schema.toString());
} catch(ClassNotFoundException e) {
System.err.println(e);
System.exit(1);
} catch(InstantiationException e) {
System.err.println(e);
System.exit(1);
} catch(IllegalAccessException e) {
System.err.println(e);
System.exit(1);
}
}
}
-Steven Willis
> -----Original Message-----
> From: Dave McAlpin [mailto:[EMAIL PROTECTED]]
> Sent: Tuesday, December 11, 2012 3:32 PM
> To: [EMAIL PROTECTED]
> Subject: RE: How to handle schema dependencies
>
> I had the same problem. My solution was to package external schema
> files into a source jar and have Maven download and extract those
> source jars at code generation time. After generation, I delete the
> external schema along with their generated code and depend on an
> external jar file at runtime.
>
> I use IDL instead of Avro schema, so this approach might not work for
> you, but here's what I did.
>
> In the external project (the one I want to import), I changed the
> pom.xml to package schemas into a source jar. Note that the fragment
> below assumes that the Avro schema is stored in src/main/schema, but
> there's nothing special about that location. Also note that I exclude
> generated Java files from the source jar.
>
> <build>
> <resources>
> <resource>
> <directory>${project.basedir}/src/main/schema</directory>
> </resource>
> </resources>
> <plugins>
> .
> .
> .
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-source-plugin</artifactId>
> <version>2.2.1</version>
> <executions>
> <execution>
> <id>attach-avdl</id>
> <phase>verify</phase>
> <goals>
> <goal>jar-no-fork</goal>
> </goals>
> <configuration>
> <includePom>true</includePom>
> <excludes>
> <exclude>**/*.java</exclude>
> </excludes>
> <includes>
> <include>*.avdl</include>
> </includes>
> </configuration>
> </execution>
> </executions>
> </plugin>
> </plugins>
> </build>
>
>
> In the project that uses the external schemas, I changed the pom.xml to
> pull in those schemas as external dependencies and delete them after
> code generation is complete. I also delete the generated java files
> that result from those external schema because I want to use the
> generated class files from an external jar rather than the locally
> generated versions.
>
> ***PLEASE NOTE*** that the code below deletes files as part of clean
I was thinking about doing something like that (adding the schemas as resources in the jars). It just seems like a lot of work for something that could be automatic. It would be nice if during schema parsing we could specify a classpath to be used for dynamic lookup of external schemas. That way Schema.parse could look up already schemas that have already been compiled like this:
package com.compete.util;
import org.apache.avro.Schema;
import org.apache.avro.generic.GenericContainer;
public class Schemas {
public static void main(String args[]) {
String name = "Date";
String space = "com.compete.avro";
try {
// If we were inside Schema.parse we would probably do:
// Class.forName(new Name(name, space).toString())
Class<?> cls = Class.forName(space+"."+name);
GenericContainer record = (GenericContainer)cls.newInstance();
Schema schema = record.getSchema();
// if we were inside Schema.parse(JsonNode schema, Names names) we
// could now just call: names.put(new Name(name, space), schema);
System.out.println(schema.toString());
} catch(ClassNotFoundException e) {
System.err.println(e);
System.exit(1);
} catch(InstantiationException e) {
System.err.println(e);
System.exit(1);
} catch(IllegalAccessException e) {
System.err.println(e);
System.exit(1);
}
}
}
-Steven Willis
> -----Original Message-----
> From: Dave McAlpin [mailto:[EMAIL PROTECTED]]
> Sent: Tuesday, December 11, 2012 3:32 PM
> To: [EMAIL PROTECTED]
> Subject: RE: How to handle schema dependencies
>
> I had the same problem. My solution was to package external schema
> files into a source jar and have Maven download and extract those
> source jars at code generation time. After generation, I delete the
> external schema along with their generated code and depend on an
> external jar file at runtime.
>
> I use IDL instead of Avro schema, so this approach might not work for
> you, but here's what I did.
>
> In the external project (the one I want to import), I changed the
> pom.xml to package schemas into a source jar. Note that the fragment
> below assumes that the Avro schema is stored in src/main/schema, but
> there's nothing special about that location. Also note that I exclude
> generated Java files from the source jar.
>
> <build>
> <resources>
> <resource>
> <directory>${project.basedir}/src/main/schema</directory>
> </resource>
> </resources>
> <plugins>
> .
> .
> .
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-source-plugin</artifactId>
> <version>2.2.1</version>
> <executions>
> <execution>
> <id>attach-avdl</id>
> <phase>verify</phase>
> <goals>
> <goal>jar-no-fork</goal>
> </goals>
> <configuration>
> <includePom>true</includePom>
> <excludes>
> <exclude>**/*.java</exclude>
> </excludes>
> <includes>
> <include>*.avdl</include>
> </includes>
> </configuration>
> </execution>
> </executions>
> </plugin>
> </plugins>
> </build>
>
>
> In the project that uses the external schemas, I changed the pom.xml to
> pull in those schemas as external dependencies and delete them after
> code generation is complete. I also delete the generated java files
> that result from those external schema because I want to use the
> generated class files from an external jar rather than the locally
> generated versions.
>
> ***PLEASE NOTE*** that the code below deletes files as part of clean
+
Doug Cutting 2012-12-11, 22:54
+
Steven Willis 2012-12-11, 23:47
+
Doug Cutting 2012-12-12, 00:25
+
Dave McAlpin 2012-12-11, 23:45
