Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> Serializing nested records

Copy link to this message
Serializing nested records
How do I serialize a record containing a nested record?  There doesn't seem
to be any way to create a DataFileWriter without tieing it to a single
record type.

Here's the scenario: I've defined an Avro schema called TransactionStatic
with nested internal record called TransactionStaticComponent:

  "namespace": "com.wily.apm.blackjack",
  "type": "record",
  "name": "TransactionStatic",
       {"name": "id", "type": "int"},
       {"name": "isIdLocal", "type": "boolean", "default": "true"},
       {"name": "contextPath",  "type": [{ "type": "array", "items":
"string" },"null"] },
       {"name": "components", "type":
           [{ "type": "array", "items" :
               [{ "type": "record",
                   "name": "TransactionStaticComponent",
                       {"name": "id", "type": "int"},
                       {"name": "isIdLocal", "type": "boolean", "default":
                       {"name": "contextPath", "type": [{ "type": "array",
"items": "string" },"null"] },
                       {"name": "application", "type" : ["string","null"]},
                       {"name": "class", "type" :  ["string","null"]},
                       {"name": "method", "type" :  ["string","null"]},
                       {"name": "lineNumber", "type" :  ["int","null"]},
                       {"name": "payload", "type": [{"type": "map",
"values": "string"},"null"] },
                       {"name": "components", "type": [{ "type": "array",
"items": "TransactionStaticComponent" }], "default": "null" }

This compiles clean and I'm able to create data in the schema.  However, if
I try to serialize a record:

DataFileWriter<TransactionStatic> staticWriter
               = new DataFileWriter<TransactionStatic>(new
ByteArrayOutputStream staticOutputStream = new ByteArrayOutputStream(1024);
staticWriter.create(TransactionStatic.SCHEMA$, staticOutputStream);

I get an Avro exception stating that TransactionStaticInstance is not

Exception in thread "main"
org.apache.avro.AvroRuntimeException: Unknown datum type:
at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:263

How can I serialize a TransactionStatic?  Also, where did the "L" come from
in "Lcom.wily..."?

View this message in context: http://apache-avro.679487.n3.nabble.com/Serializing-nested-records-tp4025992.html
Sent from the Avro - Users mailing list archive at Nabble.com.
robturner 2013-01-21, 22:29
yimca 2013-01-22, 18:34
Doug Cutting 2013-01-21, 22:04