Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> C impl, getting a schema back as json C string


Copy link to this message
-
Re: C impl, getting a schema back as json C string
That's true that you'll have to hold on to the buffer yourself.  Currently
the memory writer wraps a fixed buffer.  Alternatively, we could have a
second type of memory writer that dynamically grows in size as needed (to
say some maximum size).

Implementation details aside, what would be the ideal API/process to do
this?

-Matt
On Tue, Mar 16, 2010 at 3:02 PM, Jeff Hodges <[EMAIL PROTECTED]> wrote:

> Yeah, I got that part. The problem I'm having is getting the data back
> out of the writer. _tell just gives you the size of buffer sure, and I
> would still have to hold on to buffer myself.
> --
> Jeff
>
> On Tue, Mar 16, 2010 at 2:35 PM, Matt Massie <[EMAIL PROTECTED]> wrote:
> > Jeff-
> >
> > If you look in avro.h, you'll see the following function...
> >
> > int avro_schema_to_json(avro_schema_t schema, avro_writer_t out);
> >
> > .... which write a schema to an avro_writer_t.  This allows you to write
> the
> > schema to memory or a file... create the writer using...
> >
> > avro_writer_t avro_writer_file(FILE * fp);
> > avro_writer_t avro_writer_memory(const char *buf, int64_t len);
> >
> > I just realized that
> >
> > int64_t avro_writer_tell(avro_writer_t writer)
> >
> > is not in avro.h and it should be.  It will tell you the size of the data
> > written to the writer.
> >
> > Hope this help.  Keep the questions coming.
> >
> > -Matt
> >
> >
> >
> >
> >
> >
> >
> > On Sun, Mar 14, 2010 at 10:14 PM, Jeff Hodges <[EMAIL PROTECTED]>
> wrote:
> >> (Note, that I've passed this in memory avro_writer_t to
> >> avro_schema_to_json and I'm just trying to get the string inside of it
> >> out.)
> >> --
> >> Jeff
> >>
> >> On Sun, Mar 14, 2010 at 12:39 AM, Jeff Hodges <[EMAIL PROTECTED]>
> wrote:
> >>> I can't figure out the API for turning an avro_schema_t back into an
> >>> in-memory string of json data.
> >>>
> >>> It seems I'm supposed to get it out of an avro_writer_t that I've
> >>> created with avro_writer_memory() but I'm a little lost once we start
> >>> having to use avro_writer_to_memory() to get back the original
> >>> avro_memory_writer_t.
> >>>
> >>> Sample code?
> >>> --
> >>> Jeff
> >>>
> >>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB