Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro, mail # dev - Python-Avro Codegen Proposal


Copy link to this message
-
Re: Python-Avro Codegen Proposal
Philip Zeyliger 2013-06-04, 18:24
Hi Tal,

I would encourage you to file a JIRA and contribute your changes!

I agree that the generated code approach is a great one for things like RPC
usage, where the schema changes slowly and the extra checking is super
handy.

-- Philip
On Mon, Jun 3, 2013 at 6:28 PM, Stefan Krawczyk <[EMAIL PROTECTED]> wrote:

> Hi Tal,
>
> I'm interested in using Avro + Python. That would sound useful, any chance
> I could have a play with what you've done?
>
> Cheers,
>
> Stefan
>
>
> On Sun, Jun 2, 2013 at 10:28 PM, Tal Levy <[EMAIL PROTECTED]> wrote:
>
> > Hi,
> >
> > I recently started using Avro at my work and we found it difficult to
> keep
> > track of what python dict matched to what schema. Instead of having
> > random dicts being populated and then attempted to be serialized to
> avro, I
> > thought
> > it would be more readable and less error prone to codegen the python dict
> > for developers. These classes are type checked field by field. Although
> it
> > does not
> > have the advantage of compiled type checking like in the java codegen, it
> > is a
> > friendly wrapper around python dicts representing avro records to be
> > serialized.
> >
> > let me know what you think about this, I am still tweaking how it
> behaves.
> > I understand it is a bit unpythonic to enforce types in this way, but the
> > readability
> > is worth it nonetheless.
> >
> > here is an example record:
> > https://gist.github.com/talevy/5696236
> >
> > I extended the avro compiler/tools to provide both java and python
> codegen
> > functionality.
> > so if this sounds like something others would use, maybe it makes sense
> to
> > include it
> > into the main repo.
> >
> >
> > Thanks,
> > Tal
> >
>