Avro, mail # user - Error deserializing Avro Byte[] Data - 2014-02-17, 22:00
Solr & Elasticsearch trainings in New York & San Francisco [more info][hide]
 Search Hadoop and all its subprojects:

Switch to Threaded View
Copy link to this message
-
Error deserializing Avro Byte[] Data
Hi Everyone,

I'm currently trying to write a serializer function (shown below) for
Apache Flume that converts the Flume Avro Event to my custom Avro event.
 The issue I'm seeing is if I convert the Flume Avro byte[] data directly
using the binaryDecoder function, it does not give the correct data values.
 However, it works correctly if I convert the Flume Avro byte[] data to a
JSON record and then using the JsonDecoder.

Am I using the binaryDecoder incorrectly?

I appreciate your help!

Jonathan

protected CustomEvent convert(Event flumeEvent) {

byte[] avroByteData = flumeEvent.getBody();
SpecificDatumReader<CustomEvent> reader = new
SpecificDatumReader<CustomEvent>(this.getSchema());
String jsonRecord = new String(avroByteData, Charsets.UTF_8);

try {
JsonDecoder jsonDecoder =
DecoderFactory.get().jsonDecoder(customEvent.getSchema(), jsonRecord);
customEvent = reader.read(null, jsonDecoder);
System.out.println(customEvent.hashCode());
} catch (IOException e1) {
e1.printStackTrace();
}
try {
Decoder binaryDecoder = DecoderFactory.get().binaryDecoder(avroByteData,
null);
//singleEvent = reader.read(null, binaryDecoder);
System.out.println(reader.read(null, binaryDecoder).hashCode());
System.out.println();
} catch (IOException e) {
//logger.error("Unable to read SingleEvent record. Exception follows.", e);
}
return customEvent;

}

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB