Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> Anonymous record schemas in data files

Eric Sammer 2013-03-04, 07:50
Copy link to this message
Re: Anonymous record schemas in data files
As Francis noted, anonymous records are not permitted.  That said, the
runtime uses anonymous record schemas internally to implement message
parameter lists (which are written and read like records, but don't
have names).

How did you manage to create a file containing an anonymous record?
Perhaps the API lets you create anonymous record schemas?  If so, we
should probably fix that, so they're only created by the Protocol
parser via a package-private API.


On Sun, Mar 3, 2013 at 11:50 PM, Eric Sammer <[EMAIL PROTECTED]> wrote:
> All:
> I'm looking for some clarity on the use of anonymous records in Avro data
> files. Is this considered legal? 1.7.3 allows one to write a data file with
> DataFileWriter with an anonymous record schema that can't be read back which
> is not the nicest behavior. Here's a contrived example of a data file:
> esammer:~/ esammer$ ~/bin/avro-tool getmeta 1362381940987-1
> Exception in thread "main" org.apache.avro.SchemaParseException: No name in
> schema: {"type":"record","fields":[{"name":"word","type":"string"}]}
>         at org.apache.avro.Schema.getRequiredText(Schema.java:1198)
>         at org.apache.avro.Schema.parse(Schema.java:1066)
>         at org.apache.avro.Schema$Parser.parse(Schema.java:927)
>         at org.apache.avro.Schema$Parser.parse(Schema.java:917)
>         at org.apache.avro.Schema.parse(Schema.java:974)
>         at
> org.apache.avro.file.DataFileStream.initialize(DataFileStream.java:124)
>         at
> org.apache.avro.file.DataFileReader.<init>(DataFileReader.java:97)
>         at
> org.apache.avro.file.DataFileReader.<init>(DataFileReader.java:89)
>         at
> org.apache.avro.tool.DataFileGetMetaTool.run(DataFileGetMetaTool.java:63)
>         at org.apache.avro.tool.Main.run(Main.java:78)
>         at org.apache.avro.tool.Main.main(Main.java:67)
> Before I filed the bug I wanted to clarify that anonymous records are
> against the spec (or that they aren't, and the bug is the schema parser).
> Thanks.
> --
> Eric Sammer
> twitter: esammer
> data: www.cloudera.com
Eric Sammer 2013-03-04, 17:57
Doug Cutting 2013-03-04, 18:16
Francis Galiegue 2013-03-04, 09:34