Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> Parsing a Pair's value - inherited namespace?

Copy link to this message
Parsing a Pair's value - inherited namespace?

I noticed that after calling:

Pair.getPairSchema(Schema.create(Type.INT), schema));/
(schema is parsed from an avro file, and has no namespace)

When the M/R job is run, there's a call to /AvroJob.getJobOutputSchema
/which calls /Schema.parse/ - which parses the schema I set in

The problem: the schema (the value in the Pair returned by
getJobOutputSchema) has no namespace, so it gets it from the Pair (which is
"org.apache.avro.mapred"), so my schema's full name is

Why do I care? Well, when the schema is a union, the avro items passed to
the AvroMapper.map have no namespace (so their full name is "MySchema", and
are not matching the mapper's output schema (I get a "not in union"
exception). Note that this happens although I pass a Pair to collect(), but
in this case, the Pair's namespace is ignored.

I hope it's clear - will be happy to provide any more details.

Your help is much appreciated!!

View this message in context: http://apache-avro.679487.n3.nabble.com/Parsing-a-Pair-s-value-inherited-namespace-tp4026810.html
Sent from the Avro - Users mailing list archive at Nabble.com.