Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> Support for Serialization and Externalization?


+
Joe Gamache 2012-04-23, 17:37
Copy link to this message
-
Re: Support for Serialization and Externalization?


On 4/23/12 10:37 AM, "Joe Gamache" <[EMAIL PROTECTED]> wrote:

> Hello,
>
> We have been using  Avro successfully to serialize many of our objects, using
> binary encoding, for storage and retrieval.  Although the documentation about
> the Reflect Mapping states:
>     This API is not recommended except as a stepping stone for systems that
> currently uses Java interfaces to define RPC protocols.
> we used this mapping as that recommendation did not seem to apply.  We do not
> use the serialized data for RPC (or any other messaging system).    In fact,
> this part has in-place for a while and works exceptionally well.
>
> Now we would like to "externalize" a smaller subset of the objects for
> interaction with a WebApp.  Here we would like to use the JSON encoding and
> the "specific" mapping.    We tried having this set of objects implement
> "GenericRecord", however, this then breaks the use of Reflection on these
> objects.  [The ReflectData.createSchema method checks for this condition.]
>
> Can Avro be used to serialize objects one way, and externalize them another?
> [The externalized objects are a subset of the serialized ones.]   Perhaps more
> generally, my question is: can both binary encoding and JSON encoding be
> supported on overlapping objects using different mappers?   If yes, what is
> the best way to accomplish this?

That should be possible.  If not I think It is a bug.  The Java reflect API
is supposed to be able to handle Specific and Generic records, or at least
there is supposed to be a way to use them both.
What is the specific error, from what API call?  Perhaps it is a simple fix
and you can submit a patch and test to JIRA?

Thanks,

-Scott

>
> Thanks for any help - I am still quite a noob here so I greatly appreciate any
> additional details!
>
> Joe Gamache
+
Joe Gamache 2012-05-04, 17:59
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB