Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> AvroKeyInputFormat Writer Schema


Copy link to this message
-
AvroKeyInputFormat Writer Schema
Hi,

I am using AvroKeyInputFormat.  I have not specified a reader schema.  From
my mapper, I would like to access the schema that was used to write the
container file (i.e. the writer schema).

Is there a way to do this?

If not, is this a reasonable enhancement request?

It does look like I could use FileSplit#getPath() to access the container
file manually and pull out the schema.  This may work but it seems a shame
to do this work when it looks like the schema has already been pulled out
by AvroRecordReaderBase#createAvroFileReader(...)

Also, I suppose you can always derive the effective schema from the datum
when reading records, primitives, etc.  However, it seems difficult to
derive a schema when dealing with a list or map container file.  And, it
seems impossible when dealing with a union type container.

Thanks!
Josh
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB