Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> Rubygem Avro::DataFile issue?


Copy link to this message
-
Re: Rubygem Avro::DataFile issue?
Hi Michael,

Thanks for reporting this. Your sample code seems to work correctly on Ruby 1.9.3, but fails on Ruby 2.*. The data file that it's writing is corrupted (the Java implementation of Avro can't read it either). So that looks like a bug in Avro when writing data files on Ruby >=2.0.

I just ran Avro's test suite on Ruby 2.0 and several tests failed. It looks like you're the first person to run it on 2.0. Sorry about that!

Would you mind filing a bug report at https://issues.apache.org/jira/browse/AVRO/ and attaching your example from the gist? If you feel like submitting a patch, we'd appreciate it. If not, I'll try to look into it soon.

Martin

On 23 Apr 2014, at 16:28, Michael Ries <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
Hey. I'm trying to use the avro 1.7.5 gem (avoiding the 1.7.6 dependency bug) and I can't seem to read data back out of my data files. I am getting the error "avro-1.7.5/lib/avro/io.rb:46:in `byte!': undefined method `unpack' for nil:NilClass (NoMethodError)".

I put a more complete write up of what I am trying to do in this gist: https://gist.github.com/hqmq/11193064

Any idea what I'm doing wrong? It seems to be always failing while trying to decode the last record in the file. I get no luck on ruby 2.1.1 or 2.0.0.

I tried creating an avro file using the java client library (members.from_java_client.avro) and the rubygem can read that file just fine. It looks like the two files are different by just 16 bytes.
 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB