Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> Anonymous record schemas in data files


Copy link to this message
-
Re: Anonymous record schemas in data files
As Francis noted, anonymous records are not permitted.  That said, the
runtime uses anonymous record schemas internally to implement message
parameter lists (which are written and read like records, but don't
have names).

How did you manage to create a file containing an anonymous record?
Perhaps the API lets you create anonymous record schemas?  If so, we
should probably fix that, so they're only created by the Protocol
parser via a package-private API.

Doug

On Sun, Mar 3, 2013 at 11:50 PM, Eric Sammer <[EMAIL PROTECTED]> wrote:
> All:
>
> I'm looking for some clarity on the use of anonymous records in Avro data
> files. Is this considered legal? 1.7.3 allows one to write a data file with
> DataFileWriter with an anonymous record schema that can't be read back which
> is not the nicest behavior. Here's a contrived example of a data file:
>
> esammer:~/ esammer$ ~/bin/avro-tool getmeta 1362381940987-1
> Exception in thread "main" org.apache.avro.SchemaParseException: No name in
> schema: {"type":"record","fields":[{"name":"word","type":"string"}]}
>         at org.apache.avro.Schema.getRequiredText(Schema.java:1198)
>         at org.apache.avro.Schema.parse(Schema.java:1066)
>         at org.apache.avro.Schema$Parser.parse(Schema.java:927)
>         at org.apache.avro.Schema$Parser.parse(Schema.java:917)
>         at org.apache.avro.Schema.parse(Schema.java:974)
>         at
> org.apache.avro.file.DataFileStream.initialize(DataFileStream.java:124)
>         at
> org.apache.avro.file.DataFileReader.<init>(DataFileReader.java:97)
>         at
> org.apache.avro.file.DataFileReader.<init>(DataFileReader.java:89)
>         at
> org.apache.avro.tool.DataFileGetMetaTool.run(DataFileGetMetaTool.java:63)
>         at org.apache.avro.tool.Main.run(Main.java:78)
>         at org.apache.avro.tool.Main.main(Main.java:67)
>
> Before I filed the bug I wanted to clarify that anonymous records are
> against the spec (or that they aren't, and the bug is the schema parser).
>
> Thanks.
> --
> Eric Sammer
> twitter: esammer
> data: www.cloudera.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB