Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro, mail # user - Avro data type for datetimes?

Copy link to this message
Re: Avro data type for datetimes?
Doug Cutting 2013-08-23, 16:39
Since release 1.7.4, Avro Java will serialize and deserialize
instances of java.util.Date using the following schema:

  {"type":"string", "java-class":"java.util.Date"}

This might thus be use as a standard Date schema, with other
implementations also treating this schema specially.

Or one can devise a different standard, common schema for datetimes,
perhaps something like:

  {"type":"record", "name":"org.apache.avro.Datetime",
"fields":[{"name":"millisSinceEpoch", "type":"long"}]}

Schema parsers might then be configured to pre-define this, so that
folks can simply refer to it with "type":"org.apache.avro.Datetime".
(For example, AVRO-1188 permits one to specify directories containing
schemas to include when Maven compiles schemas, protocols and IDL.)
For back-compatibility however, a schema parser must not be required
to include it.  When a schema is printed, e.g., in a data file or in
an RPC handshake, the full schema will still be printed, so that
parsers that do not have this pre-defined can still process the data.


On Fri, Aug 23, 2013 at 7:48 AM, Andrew Pennebaker
> Could there be an Avro data type for date times? That would greatly increase
> data compatibility between systems, reducing errors from string printing and
> parsing, and relaxing the need for producers and consumers to agree on
> specific formatters.