Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> Versioning of an array of a record


Copy link to this message
-
Re: Versioning of an array of a record
On 09/16/2010 09:25 AM, Scott Carey wrote:
> Generally, Avro recommends storing the schema with the data.   For a
> file that means in the header of the file, for a key/value store that
> means in some system metadata.   Any individual store can only keep
> data serialized with one schema.

Another good pattern is to store the hashcode of the writer's schema
with each written instance, then keep written schemas in a separate
store, keyed by hashcode.  For example, Sam Pullara's done this in his
HAvroBase:

http://www.javarants.com/2010/06/30/havrobase-a-searchable-evolvable-entity-store-on-top-of-hbase-and-solr/

Doug
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB