Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # dev >> Dynamic Schema


Copy link to this message
-
Dynamic Schema
I have very dynamic data that i want to write to an avro file. The solution
i have is to store all that data in the memory and then calculate the
schema, and then start the writing.

This causes the files to be smaller in size, because of the memory
limitations.

What i am looking for is that i will start data as and when it is
collected, but how should i compute the schema in this case? Can i change
the schema for an avro file?

Thanks
Amit

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB