Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> Avro Schema to SQL


Copy link to this message
-
Re: Avro Schema to SQL
Not all Avro schemas can be converted to SQL.  Primarily, Unions can pose
challenges, as well as recursive references.

Nested types are a mixed bag ‹ some SQL-related systems have rich support
for nested types and/or JSON (e.g. PosgtgreSQL) which can make this easier,
while others are more crude (MySQL, HIVE).

With Unions, in some cases a union field can be expanded/flattened into
multiple fields, of which only one is not null.  Recursive types can be
transformed into key references.

In general, all of these transformation strategies require decisions by the
user and potentially custom work depending on what database is involved.

Traversing an Avro Schema in Java is done via the Schema API, the Javadoc
explains it and there are many examples in the avro source code.  The type
of schema must be checked, and for each nested type a different decent into
its contained types can occur.

From:  Avinash Dongre <[EMAIL PROTECTED]>
Reply-To:  "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
Date:  Wednesday, June 19, 2013 2:31 AM
To:  "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
Subject:  Avro Schema to SQL

Is there know tool/framework available to convert Avro Schema into SQL.
If now , How Do i iterate over the schema to find out what records, enums
are there. I can think of how to achieve this with simple Schema, but I am
not able to figure out a way for nested schemas.

Thanks
Avinash

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB