Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> ReflectDatumReader and Writer classes


+
Peter Cameron 2012-07-05, 16:44
+
Mark Hayes 2012-07-05, 17:09
+
Peter Cameron 2012-07-05, 17:18
Copy link to this message
-
Re: ReflectDatumReader and Writer classes
Regardless of how we define "dynamic" that statement in the
documentation is confusing.  Folks do find Avro reflection useful in
many cases and we should improve that statement.

Perhaps we should instead just say something like: "Transparently
supports simple classes.  See below for details and limitations."

Would that be better?

Thanks,

Doug

On Thu, Jul 5, 2012 at 10:18 AM, Peter Cameron
<[EMAIL PROTECTED]> wrote:
> Let me explain further. Our data is not static. We do not know the type of
> Java object at runtime, as we only have the schema. We use the avro reflect
> package to transparently serialise and deserialise an Object instance given
> its schema. Ours is a black box that can serialise and deserialise any
> Object given a schema. We are given the Object to serialise by the caller,
> which is not under our control -- the only constraint is that both sides
> have the schema. The Specific readers/writers need code generation, and the
> generic readers and writers expect the objects to be "indexed records" and
> so barf. For any old POJO (with schema), the black box method can only be
> satisified by Avro's reflect package, unless I'm mistaken?
>
> Peter
>
>
>
> On 05/07/2012 18:09, Mark Hayes wrote:
>
> On Thu, Jul 5, 2012 at 9:44 AM, Peter Cameron <[EMAIL PROTECTED]>
> wrote:
>>
>> "This API is not recommended except as a stepping stone for systems that
>> currently uses Java interfaces to define RPC protocols. For new RPC systems,
>> the specific API is preferred. For systems that process dynamic data, the
>> generic API is probably best."
>>
>> What I'm confused by is the assertion that the generic API is "probably
>> best" for processing dynamic data.
>
>
> I am still fairly new to Avro but I think what the warning in the docs is
> trying to say is that the Specific API is better for static data, because
> reflection is slower.  If you're representing data using a Java bean, then
> your data is static (known at build time).
>
> --mark
>
>
>
> --
> Peter Cameron
> 2iC Limited
> T: +44 (0) 208 123 7479
> E: [EMAIL PROTECTED]
> W: www.2iCworld.com
+
Peter Cameron 2012-07-05, 18:01
+
Mark Hayes 2012-07-05, 18:26
+
Doug Cutting 2012-07-05, 18:18