Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> Parsing a Pair's value - inherited namespace?


Copy link to this message
-
Re: Parsing a Pair's value - inherited namespace?
This looks to be a bug.  I filed a Jira issue and attached a patch at:

https://issues.apache.org/jira/browse/AVRO-1295

Please let me know if this fixes things for you.

Thanks,

Doug
On Tue, Apr 9, 2013 at 9:02 AM, nir_zamir <[EMAIL PROTECTED]> wrote:

> Hi,
>
> I noticed that after calling:
>
> /AvroJob.setMapOutputSchema(conf,
> Pair.getPairSchema(Schema.create(Type.INT), schema));/
> (schema is parsed from an avro file, and has no namespace)
>
> When the M/R job is run, there's a call to /AvroJob.getJobOutputSchema
> /which calls /Schema.parse/ - which parses the schema I set in
> setMapOutputSchema.
>
> The problem: the schema (the value in the Pair returned by
> getJobOutputSchema) has no namespace, so it gets it from the Pair (which is
> "org.apache.avro.mapred"), so my schema's full name is
> "org.apache.avro.mapred.MySchema".
>
> Why do I care? Well, when the schema is a union, the avro items passed to
> the AvroMapper.map have no namespace (so their full name is "MySchema", and
> are not matching the mapper's output schema (I get a "not in union"
> exception). Note that this happens although I pass a Pair to collect(), but
> in this case, the Pair's namespace is ignored.
>
> I hope it's clear - will be happy to provide any more details.
>
> Your help is much appreciated!!
>
> Thanks,
> Nir
>
>
>
>
> --
> View this message in context:
> http://apache-avro.679487.n3.nabble.com/Parsing-a-Pair-s-value-inherited-namespace-tp4026810.html
> Sent from the Avro - Users mailing list archive at Nabble.com.
>