Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro, mail # user - C impl, getting a schema back as json C string


Copy link to this message
-
Re: C impl, getting a schema back as json C string
Matt Massie 2010-03-16, 22:50
That's true that you'll have to hold on to the buffer yourself.  Currently
the memory writer wraps a fixed buffer.  Alternatively, we could have a
second type of memory writer that dynamically grows in size as needed (to
say some maximum size).

Implementation details aside, what would be the ideal API/process to do
this?

-Matt
On Tue, Mar 16, 2010 at 3:02 PM, Jeff Hodges <[EMAIL PROTECTED]> wrote:

> Yeah, I got that part. The problem I'm having is getting the data back
> out of the writer. _tell just gives you the size of buffer sure, and I
> would still have to hold on to buffer myself.
> --
> Jeff
>
> On Tue, Mar 16, 2010 at 2:35 PM, Matt Massie <[EMAIL PROTECTED]> wrote:
> > Jeff-
> >
> > If you look in avro.h, you'll see the following function...
> >
> > int avro_schema_to_json(avro_schema_t schema, avro_writer_t out);
> >
> > .... which write a schema to an avro_writer_t.  This allows you to write
> the
> > schema to memory or a file... create the writer using...
> >
> > avro_writer_t avro_writer_file(FILE * fp);
> > avro_writer_t avro_writer_memory(const char *buf, int64_t len);
> >
> > I just realized that
> >
> > int64_t avro_writer_tell(avro_writer_t writer)
> >
> > is not in avro.h and it should be.  It will tell you the size of the data
> > written to the writer.
> >
> > Hope this help.  Keep the questions coming.
> >
> > -Matt
> >
> >
> >
> >
> >
> >
> >
> > On Sun, Mar 14, 2010 at 10:14 PM, Jeff Hodges <[EMAIL PROTECTED]>
> wrote:
> >> (Note, that I've passed this in memory avro_writer_t to
> >> avro_schema_to_json and I'm just trying to get the string inside of it
> >> out.)
> >> --
> >> Jeff
> >>
> >> On Sun, Mar 14, 2010 at 12:39 AM, Jeff Hodges <[EMAIL PROTECTED]>
> wrote:
> >>> I can't figure out the API for turning an avro_schema_t back into an
> >>> in-memory string of json data.
> >>>
> >>> It seems I'm supposed to get it out of an avro_writer_t that I've
> >>> created with avro_writer_memory() but I'm a little lost once we start
> >>> having to use avro_writer_to_memory() to get back the original
> >>> avro_memory_writer_t.
> >>>
> >>> Sample code?
> >>> --
> >>> Jeff
> >>>
> >>
>