Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro, mail # user - Failed Serializing Bytes


+
Milind Vaidya 2013-04-09, 14:12
+
Douglas Creager 2013-04-11, 12:32
Copy link to this message
-
Re: Failed Serializing Bytes
Milind Vaidya 2013-04-12, 04:01
Thanks Doug. I figured that out based on your previous mails regarding
record data serialization.

I appreciate your help. Avro is interesting.
On Thu, Apr 11, 2013 at 7:32 AM, Douglas Creager <[EMAIL PROTECTED]>wrote:

> > //Data to be written
> >                 unsigned char buffer_data[] = {0x12, 0x34, 0x56,
> > 0x78,0x12,0x34,0x56,0x78,0x12, 0x34, 0x56, 0x78,0x12, 0x34, 0x56,
> > 0x78,0x12,0x34,0x56,0x78,0x12, 0x34, 0x56, 0x78,0x12, 0x34, 0x56,
> > 0x78,0x12,0x34,0x56,0x78,0x12, 0x34, 0x56, 0x78};
> >
> >                 //Serialize bytes
> >                 avro_value_t value;
> >                 avro_file_writer_t writer;
> >                 avro_schema_t byteSchema = avro_schema_bytes();
> >
> >
> >                avro_value_set_bytes(&value, buffer_data,
> > sizeof(buffer_data));
> >                 avro_file_writer_create("./resource/Data.ser",
> > byteSchema, &writer);
> >                 avro_file_writer_append_value(writer, &value);
> >                 avro_file_writer_flush(writer);
> >                 avro_file_writer_close(writer);
> >
> > is this correct way to serialize bytes ? If yes not working for me :(
>
> The file_writer part looks right; I think the problem is where you're
> filling in the value before serializing it.  Think of your avro_value_t
> instance as a pointer — it doesn't look like one, since you can't see a
> "*", but under the covers it is.  You call avro_value_set_bytes, but you
> haven't initialized "value" to anything, so this is equivalent to
> dereferencing a NULL or undefined pointer.
>
> You'll need to allocate an actual value instance, and have your "value"
> variable point at the new instance.  Then your set_bytes call will succeed:
>
>     avro_value_iface_t  *iface;
>     iface = avro_generic_class_from_schema(byteSchema);
>     avro_generic_value_new(iface, &value);
>
> In the new value API, there's an interface (avro_value_iface_t) that is
> implemented by a handful of different types internally.  And it's also
> possible to write your own implementations — to make an avro_value_t
> read directly from an existing C struct, for instance.  The
> "avro_generic" implementation is one that works with any schema, and
> allocates the right amount of space (in the avro_generic_value_new call)
> to hold instances of the schema in question (which is provided to the
> avro_generic_class_from_schema call).
>
>