Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> Avro serialization in Ruby - how to write to a String/buffer instead of file


+
Abishek Baskaran 2014-02-11, 04:35
Copy link to this message
-
Re: Avro serialization in Ruby - how to write to a String/buffer instead of file
Hi Abishek,

You got most of the way there! Next thing you need to do is to use the
DatumWriter you created:

dw.write({'name': 'Bob', 'id': 42, 'city': 'New York'}, encoder)
buffer.string # now contains the binary representation of that record

Pro tip: in Ruby >= 1.9, if you do StringIO.new it uses your default
character encoding, which may be UTF-8 or something like that. However,
Avro generates binary data, not UTF-8, so that may cause errors down the
line. Safest if you tell Ruby explicitly that your StringIO is going to
contain binary data. You can do that as follows:

buffer = StringIO.new("".force_encoding("BINARY"))

We should really make a simper API for this!

Best,
Martin
On 11 February 2014 04:34, Abishek Baskaran <[EMAIL PROTECTED]> wrote:
 
+
Abishek Baskaran 2014-02-11, 21:50
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB