Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> Embedded Record types using GenericRecord


Copy link to this message
-
Embedded Record types using GenericRecord
Hi,

I have a schema that looks like this:

{
    "name" : "identified_wrapper",
    "type" : "record",
    "fields" : [
         {"name" : "uuid", "type" : "string"},
        {"name" : "filename", "type":"string"},
        {"name" : "content_hash", "type":"string"},
        {"name" : "body", "type":["bytes", "null"]},
        {"name" : "mime_type", "type": ["string", "null"]},
        {"name" : "type_specific", "type" : [{
                                                "name" : "message_meta_data",
                                                "type" : "record",
                                                "fields" : [
                                                   {"name" : "subject", "type" : ["string", "null"]}]
                                             },
                                             {
                                                "name" : "dublin_core_meta_data",
                                                "type" : "record",
                                                "fields" : [
                                                   {"name" : "subject_one", "type" : ["string", "null"]}]
                                             },
                                             {
                                                "name" : "image_meta_data",
                                                "type" : "record",
                                                "fields" : [{"name" : "subject_two", "type" : ["string", "null"]}]
                                              },
                                              "null"
                                            ]}
    ]
}
I'm trying to populate this object but I can't figure out how to create records for the type_specific union type using GenericRecord.  My code currently looks like :

    public static GenericRecord populateRecord(Schema schema) throws Exception {
            GenericRecord record = new GenericData.Record(schema);

            String uuid = generateUuid();
            record.put(UUID, uuid);
            record.put(FILENAME, "TEST");
            record.put(CONTENT_HASH, "797987");
            record.put(BODY, null);
            record.put(MIME_TYPE, "123321");

            GenericRecord record_dc = new GenericData.Record(schema);
            record_dc.put("subject", null);
            record.put("type_specific", record_dc);
            return record;
        }
This returns with the error :

Exception in thread "main" org.apache.avro.AvroRuntimeException: Not a valid schema field: subject
at org.apache.avro.generic.GenericData$Record.put(GenericData.java:90)
at org.iqt.cdl.SpecificMetadataAvroExample.populateRecord(SpecificMetadataAvroExample.java:79)
at org.iqt.cdl.SpecificMetadataAvroExample.main(SpecificMetadataAvroExample.java:133)
Is there  a way to do this?

Karthik
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB