Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> Avro serialization in Ruby - how to write to a String/buffer instead of file


+
Abishek Baskaran 2014-02-11, 04:35
Copy link to this message
-
Re: Avro serialization in Ruby - how to write to a String/buffer instead of file
Hi Abishek,

You got most of the way there! Next thing you need to do is to use the
DatumWriter you created:

dw.write({'name': 'Bob', 'id': 42, 'city': 'New York'}, encoder)
buffer.string # now contains the binary representation of that record

Pro tip: in Ruby >= 1.9, if you do StringIO.new it uses your default
character encoding, which may be UTF-8 or something like that. However,
Avro generates binary data, not UTF-8, so that may cause errors down the
line. Safest if you tell Ruby explicitly that your StringIO is going to
contain binary data. You can do that as follows:

buffer = StringIO.new("".force_encoding("BINARY"))

We should really make a simper API for this!

Best,
Martin
On 11 February 2014 04:34, Abishek Baskaran <[EMAIL PROTECTED]> wrote:
 
+
Abishek Baskaran 2014-02-11, 21:50