Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # dev >> C library


+
Douglas Creager 2010-10-14, 01:45
+
Bruce Mitchener 2010-10-14, 02:01
+
Douglas Creager 2010-10-14, 02:43
+
Bruce Mitchener 2010-10-14, 02:56
> Not to me. :)  I'm assuming that you mean something that uses GValue and so
> on?

Ah, whoops.  No, I'm not suggesting GValue.  *shudder*

I was thinking more like using:

 • GObject for the schema/datum subclassing
 • GHashTable or GTree to store a record schema's fields, etc.
 • GIO for the generic I/O interfaces
 • GQuark instead of the atom implementation that was checked in and
   then reverted

> I don't want the overhead of that sort of thing at all in my C code.  I'm
> supporting resource constrained platforms, so I just want to go from my C
> struct straight to a buffer without building an intermediate data structure.

We're in violent agreement.  One thing I've started experimenting with
is a “streaming” API, so that instead of creating a tree of avro_datum_t
instances, the file reader calls a series of callback functions as each
bit of data is encountered.  We're generating Avro files from an
existing C network sensor application, and it's a bit of overhead (in
both code and speed) to have to move between our actual data types and
the avro_datum_t instances.

–doug

+
Bruce Mitchener 2010-10-14, 03:58
+
Matt Massie 2010-10-14, 15:59
+
Douglas Creager 2010-10-15, 22:02
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB