Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> org.apache.avro.SchemaParseException using avro-1.7.6.jar


Copy link to this message
-
org.apache.avro.SchemaParseException using avro-1.7.6.jar
Hi,

I am trying to read *two* avro files using Pig AvroStorage. The records in
each file have the same schema but files differ slightly in the schema.
Schema in the second file is an extension of the schema in the first file. I
am providing "MULTIPLE_SCHEMAS" option to AvroStorage to merge the two
schemas but I get the following error

Failed to parse: Can't redefine: merged schema (generated by
AvroStorage).merged
at
org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:198)
at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1676)
at org.apache.pig.PigServer$Graph.access$000(PigServer.java:1409)
at org.apache.pig.PigServer.parseAndBuild(PigServer.java:342)
at org.apache.pig.PigServer.executeBatch(PigServer.java:367)
at org.apache.pig.PigServer.executeBatch(PigServer.java:353)
at
org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:140)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:202)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:173)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
at org.apache.pig.Main.run(Main.java:607)
at org.apache.pig.Main.main(Main.java:156)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: org.apache.avro.SchemaParseException: Can't redefine: merged
schema (generated by AvroStorage).merged
at org.apache.avro.Schema$Names.put(Schema.java:1019)
at org.apache.avro.Schema$NamedSchema.writeNameRef(Schema.java:496)
at org.apache.avro.Schema$RecordSchema.toJson(Schema.java:611)
at org.apache.avro.Schema$ArraySchema.toJson(Schema.java:722)
at org.apache.avro.Schema$RecordSchema.fieldsToJson(Schema.java:633)
at org.apache.avro.Schema$RecordSchema.toJson(Schema.java:620)
at org.apache.avro.Schema.toString(Schema.java:291)
at org.apache.avro.Schema.toString(Schema.java:281)
at java.lang.String.valueOf(String.java:2854)
at java.lang.StringBuilder.append(StringBuilder.java:128)
at
org.apache.pig.piggybank.storage.avro.AvroStorage.getSchema(AvroStorage.java:370)
at
org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:175)
at org.apache.pig.newplan.logical.relational.LOLoad.<init>(LOLoad.java:89)
at
org.apache.pig.parser.LogicalPlanBuilder.buildLoadOp(LogicalPlanBuilder.java:853)
at
org.apache.pig.parser.LogicalPlanGenerator.load_clause(LogicalPlanGenerator.java:3479)
at
org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1536)
at
org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1013)
at
org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:553)
at
org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:421)
at
org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:188)
... 16 more
Its using avro-1.7.6.jar. Is there a way to understand what would be causing
this?

Thanks,

View this message in context: http://apache-avro.679487.n3.nabble.com/org-apache-avro-SchemaParseException-using-avro-1-7-6-jar-tp4030220.html
Sent from the Avro - Users mailing list archive at Nabble.com.