Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> Avro data type for datetimes?


Copy link to this message
-
Re: Avro data type for datetimes?
Since release 1.7.4, Avro Java will serialize and deserialize
instances of java.util.Date using the following schema:

  {"type":"string", "java-class":"java.util.Date"}

This might thus be use as a standard Date schema, with other
implementations also treating this schema specially.

Or one can devise a different standard, common schema for datetimes,
perhaps something like:

  {"type":"record", "name":"org.apache.avro.Datetime",
"fields":[{"name":"millisSinceEpoch", "type":"long"}]}

Schema parsers might then be configured to pre-define this, so that
folks can simply refer to it with "type":"org.apache.avro.Datetime".
(For example, AVRO-1188 permits one to specify directories containing
schemas to include when Maven compiles schemas, protocols and IDL.)
For back-compatibility however, a schema parser must not be required
to include it.  When a schema is printed, e.g., in a data file or in
an RPC handshake, the full schema will still be printed, so that
parsers that do not have this pre-defined can still process the data.

Doug

On Fri, Aug 23, 2013 at 7:48 AM, Andrew Pennebaker
<[EMAIL PROTECTED]> wrote:
> Could there be an Avro data type for date times? That would greatly increase
> data compatibility between systems, reducing errors from string printing and
> parsing, and relaxing the need for producers and consumers to agree on
> specific formatters.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB