Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> Error deserializing Avro Byte[] Data


Copy link to this message
-
Error deserializing Avro Byte[] Data
Hi Everyone,

I'm currently trying to write a serializer function (shown below) for
Apache Flume that converts the Flume Avro Event to my custom Avro event.
 The issue I'm seeing is if I convert the Flume Avro byte[] data directly
using the binaryDecoder function, it does not give the correct data values.
 However, it works correctly if I convert the Flume Avro byte[] data to a
JSON record and then using the JsonDecoder.

Am I using the binaryDecoder incorrectly?

I appreciate your help!

Jonathan

protected CustomEvent convert(Event flumeEvent) {

byte[] avroByteData = flumeEvent.getBody();
SpecificDatumReader<CustomEvent> reader = new
SpecificDatumReader<CustomEvent>(this.getSchema());
String jsonRecord = new String(avroByteData, Charsets.UTF_8);

try {
JsonDecoder jsonDecoder =
DecoderFactory.get().jsonDecoder(customEvent.getSchema(), jsonRecord);
customEvent = reader.read(null, jsonDecoder);
System.out.println(customEvent.hashCode());
} catch (IOException e1) {
e1.printStackTrace();
}
try {
Decoder binaryDecoder = DecoderFactory.get().binaryDecoder(avroByteData,
null);
//singleEvent = reader.read(null, binaryDecoder);
System.out.println(reader.read(null, binaryDecoder).hashCode());
System.out.println();
} catch (IOException e) {
//logger.error("Unable to read SingleEvent record. Exception follows.", e);
}
return customEvent;

}

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB