Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> Parsing a Pair's value - inherited namespace?


+
nir_zamir 2013-04-09, 16:02
Copy link to this message
-
Re: Parsing a Pair's value - inherited namespace?
This looks to be a bug.  I filed a Jira issue and attached a patch at:

https://issues.apache.org/jira/browse/AVRO-1295

Please let me know if this fixes things for you.

Thanks,

Doug
On Tue, Apr 9, 2013 at 9:02 AM, nir_zamir <[EMAIL PROTECTED]> wrote:

> Hi,
>
> I noticed that after calling:
>
> /AvroJob.setMapOutputSchema(conf,
> Pair.getPairSchema(Schema.create(Type.INT), schema));/
> (schema is parsed from an avro file, and has no namespace)
>
> When the M/R job is run, there's a call to /AvroJob.getJobOutputSchema
> /which calls /Schema.parse/ - which parses the schema I set in
> setMapOutputSchema.
>
> The problem: the schema (the value in the Pair returned by
> getJobOutputSchema) has no namespace, so it gets it from the Pair (which is
> "org.apache.avro.mapred"), so my schema's full name is
> "org.apache.avro.mapred.MySchema".
>
> Why do I care? Well, when the schema is a union, the avro items passed to
> the AvroMapper.map have no namespace (so their full name is "MySchema", and
> are not matching the mapper's output schema (I get a "not in union"
> exception). Note that this happens although I pass a Pair to collect(), but
> in this case, the Pair's namespace is ignored.
>
> I hope it's clear - will be happy to provide any more details.
>
> Your help is much appreciated!!
>
> Thanks,
> Nir
>
>
>
>
> --
> View this message in context:
> http://apache-avro.679487.n3.nabble.com/Parsing-a-Pair-s-value-inherited-namespace-tp4026810.html
> Sent from the Avro - Users mailing list archive at Nabble.com.
>
+
nir_zamir 2013-04-14, 08:35
+
Doug Cutting 2013-04-16, 23:27
+
nir_zamir 2013-04-18, 07:46
+
nir_zamir 2013-04-25, 12:22
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB