Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # dev >> C library

Douglas Creager 2010-10-14, 01:45
Bruce Mitchener 2010-10-14, 02:01
Douglas Creager 2010-10-14, 02:43
Bruce Mitchener 2010-10-14, 02:56
> Not to me. :)  I'm assuming that you mean something that uses GValue and so
> on?

Ah, whoops.  No, I'm not suggesting GValue.  *shudder*

I was thinking more like using:

 • GObject for the schema/datum subclassing
 • GHashTable or GTree to store a record schema's fields, etc.
 • GIO for the generic I/O interfaces
 • GQuark instead of the atom implementation that was checked in and
   then reverted

> I don't want the overhead of that sort of thing at all in my C code.  I'm
> supporting resource constrained platforms, so I just want to go from my C
> struct straight to a buffer without building an intermediate data structure.

We're in violent agreement.  One thing I've started experimenting with
is a “streaming” API, so that instead of creating a tree of avro_datum_t
instances, the file reader calls a series of callback functions as each
bit of data is encountered.  We're generating Avro files from an
existing C network sensor application, and it's a bit of overhead (in
both code and speed) to have to move between our actual data types and
the avro_datum_t instances.


Bruce Mitchener 2010-10-14, 03:58
Matt Massie 2010-10-14, 15:59
Douglas Creager 2010-10-15, 22:02