Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # dev >> Python-Avro Codegen Proposal


Copy link to this message
-
Re: Python-Avro Codegen Proposal
Hi Tal,

I would encourage you to file a JIRA and contribute your changes!

I agree that the generated code approach is a great one for things like RPC
usage, where the schema changes slowly and the extra checking is super
handy.

-- Philip
On Mon, Jun 3, 2013 at 6:28 PM, Stefan Krawczyk <[EMAIL PROTECTED]> wrote:

> Hi Tal,
>
> I'm interested in using Avro + Python. That would sound useful, any chance
> I could have a play with what you've done?
>
> Cheers,
>
> Stefan
>
>
> On Sun, Jun 2, 2013 at 10:28 PM, Tal Levy <[EMAIL PROTECTED]> wrote:
>
> > Hi,
> >
> > I recently started using Avro at my work and we found it difficult to
> keep
> > track of what python dict matched to what schema. Instead of having
> > random dicts being populated and then attempted to be serialized to
> avro, I
> > thought
> > it would be more readable and less error prone to codegen the python dict
> > for developers. These classes are type checked field by field. Although
> it
> > does not
> > have the advantage of compiled type checking like in the java codegen, it
> > is a
> > friendly wrapper around python dicts representing avro records to be
> > serialized.
> >
> > let me know what you think about this, I am still tweaking how it
> behaves.
> > I understand it is a bit unpythonic to enforce types in this way, but the
> > readability
> > is worth it nonetheless.
> >
> > here is an example record:
> > https://gist.github.com/talevy/5696236
> >
> > I extended the avro compiler/tools to provide both java and python
> codegen
> > functionality.
> > so if this sounds like something others would use, maybe it makes sense
> to
> > include it
> > into the main repo.
> >
> >
> > Thanks,
> > Tal
> >
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB