Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro, mail # user - AvroKeyInputFormat Writer Schema


Copy link to this message
-
AvroKeyInputFormat Writer Schema
Josh Spiegel 2013-02-27, 16:55
Hi,

I am using AvroKeyInputFormat.  I have not specified a reader schema.  From
my mapper, I would like to access the schema that was used to write the
container file (i.e. the writer schema).

Is there a way to do this?

If not, is this a reasonable enhancement request?

It does look like I could use FileSplit#getPath() to access the container
file manually and pull out the schema.  This may work but it seems a shame
to do this work when it looks like the schema has already been pulled out
by AvroRecordReaderBase#createAvroFileReader(...)

Also, I suppose you can always derive the effective schema from the datum
when reading records, primitives, etc.  However, it seems difficult to
derive a schema when dealing with a list or map container file.  And, it
seems impossible when dealing with a union type container.

Thanks!
Josh
+
Doug Cutting 2013-02-27, 18:02
+
Josh Spiegel 2013-02-27, 19:53