-
AvroKeyInputFormat Writer SchemaJosh Spiegel 2013-02-27, 16:55
Hi,
I am using AvroKeyInputFormat. I have not specified a reader schema. From
my mapper, I would like to access the schema that was used to write the
container file (i.e. the writer schema).
Is there a way to do this?
If not, is this a reasonable enhancement request?
It does look like I could use FileSplit#getPath() to access the container
file manually and pull out the schema. This may work but it seems a shame
to do this work when it looks like the schema has already been pulled out
by AvroRecordReaderBase#createAvroFileReader(...)
Also, I suppose you can always derive the effective schema from the datum
when reading records, primitives, etc. However, it seems difficult to
derive a schema when dealing with a list or map container file. And, it
seems impossible when dealing with a union type container.
Thanks!
Josh
I am using AvroKeyInputFormat. I have not specified a reader schema. From
my mapper, I would like to access the schema that was used to write the
container file (i.e. the writer schema).
Is there a way to do this?
If not, is this a reasonable enhancement request?
It does look like I could use FileSplit#getPath() to access the container
file manually and pull out the schema. This may work but it seems a shame
to do this work when it looks like the schema has already been pulled out
by AvroRecordReaderBase#createAvroFileReader(...)
Also, I suppose you can always derive the effective schema from the datum
when reading records, primitives, etc. However, it seems difficult to
derive a schema when dealing with a list or map container file. And, it
seems impossible when dealing with a union type container.
Thanks!
Josh
+
Doug Cutting 2013-02-27, 18:02
+
Josh Spiegel 2013-02-27, 19:53
