Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro, mail # user - Support for Serialization and Externalization?


+
Joe Gamache 2012-04-23, 17:37
Copy link to this message
-
Re: Support for Serialization and Externalization?
Scott Carey 2012-05-01, 23:30


On 4/23/12 10:37 AM, "Joe Gamache" <[EMAIL PROTECTED]> wrote:

> Hello,
>
> We have been using  Avro successfully to serialize many of our objects, using
> binary encoding, for storage and retrieval.  Although the documentation about
> the Reflect Mapping states:
>     This API is not recommended except as a stepping stone for systems that
> currently uses Java interfaces to define RPC protocols.
> we used this mapping as that recommendation did not seem to apply.  We do not
> use the serialized data for RPC (or any other messaging system).    In fact,
> this part has in-place for a while and works exceptionally well.
>
> Now we would like to "externalize" a smaller subset of the objects for
> interaction with a WebApp.  Here we would like to use the JSON encoding and
> the "specific" mapping.    We tried having this set of objects implement
> "GenericRecord", however, this then breaks the use of Reflection on these
> objects.  [The ReflectData.createSchema method checks for this condition.]
>
> Can Avro be used to serialize objects one way, and externalize them another?
> [The externalized objects are a subset of the serialized ones.]   Perhaps more
> generally, my question is: can both binary encoding and JSON encoding be
> supported on overlapping objects using different mappers?   If yes, what is
> the best way to accomplish this?

That should be possible.  If not I think It is a bug.  The Java reflect API
is supposed to be able to handle Specific and Generic records, or at least
there is supposed to be a way to use them both.
What is the specific error, from what API call?  Perhaps it is a simple fix
and you can submit a patch and test to JIRA?

Thanks,

-Scott

>
> Thanks for any help - I am still quite a noob here so I greatly appreciate any
> additional details!
>
> Joe Gamache
+
Joe Gamache 2012-05-04, 17:59