Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # dev >> Determine AVRO message size

Copy link to this message
Determine AVRO message size
Hi AVRO Dev Team

We intend to use AVRO as conversion tool to send messages between C++, .NET, and JAVA applications, where actual transport is done via TCP. I am looking for a method to get actual AVRO message size after encoding is done.

Code example:

// some function here
uint8_t* buffer[1024];

// framing code here

// very short and simple message just one integer
unsigned int message = 12345;

//AVRO encoding
std::auto_ptr<avro::OutputStream> out = avro::memoryOutputStream(100);
avro::EncodePtr e = avro::binaryEncoder();
avro::encode(*e, message);
std::size_t length = out->bytesCount(); // returns 100 bytes instead of 4

std::auto_ptr<avro::InputStream> in = avro::memoryInputStream(*out);
avro::StreamReader sr(*in);
sr.readBytes(&buffer[offset], length); // will read a 100 bytes instead of 4

socket_send(socket_id, buffer, length + offset); // will send 100 bytes instead of 4

Could you advise how one can figure out exact AVRO message length? It seems currently it is just a number of buffers product buffer length. Our messages are small and do not exceed 128 bytes.

Vassili Golikov
Morgan Stanley | Institutional Equity Division
1585 Broadway, 6th Floor | New York, NY  10036
Phone: +1 212 296-4568
Be carbon conscious. Please consider our environment before printing this email.
NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies and notify the sender immediately. Mistransmission is not intended to waive confidentiality or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link: http://www.morganstanley.com/disclaimers <http://www.morganstanley.com/disclaimers>  If you cannot access these links, please notify us by reply message and we will send the contents to you. By communicating with Morgan Stanley you consent to the foregoing and to the voice recording of conversations with personnel of Morgan Stanley.