Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> ReflectDatumReader and Writer classes


Copy link to this message
-
Re: ReflectDatumReader and Writer classes
Sounds like the word "simple" is confusing!  So instead how about we
just say, "See below for details and limitations"?

Doug

On Thu, Jul 5, 2012 at 11:01 AM, Peter Cameron
<[EMAIL PROTECTED]> wrote:
> That sounds fine to me. As long as "simple" is clearly defined by the
> "details and limitations".
>
> To us, such an object means a JavaBean with properties that are either
> primitive (where primitive maps to the Avro concept of primitive), or other
> JavaBeans. Since Avro is used for transmission over the wire, it's rather
> akin to a DTO. (We actually refer to these objects in code as "complex"
> because they are values that are not Avro primitives.)
>
> cheers,
> Peter
>
>
>
> On 05/07/2012 18:42, Doug Cutting wrote:
>>
>> Regardless of how we define "dynamic" that statement in the
>> documentation is confusing.  Folks do find Avro reflection useful in
>> many cases and we should improve that statement.
>>
>> Perhaps we should instead just say something like: "Transparently
>> supports simple classes.  See below for details and limitations."
>>
>> Would that be better?
>>
>> Thanks,
>>
>> Doug
>>
>> On Thu, Jul 5, 2012 at 10:18 AM, Peter Cameron
>> <[EMAIL PROTECTED]> wrote:
>>>
>>> Let me explain further. Our data is not static. We do not know the type
>>> of
>>> Java object at runtime, as we only have the schema. We use the avro
>>> reflect
>>> package to transparently serialise and deserialise an Object instance
>>> given
>>> its schema. Ours is a black box that can serialise and deserialise any
>>> Object given a schema. We are given the Object to serialise by the
>>> caller,
>>> which is not under our control -- the only constraint is that both sides
>>> have the schema. The Specific readers/writers need code generation, and
>>> the
>>> generic readers and writers expect the objects to be "indexed records"
>>> and
>>> so barf. For any old POJO (with schema), the black box method can only be
>>> satisified by Avro's reflect package, unless I'm mistaken?
>>>
>>> Peter
>>>
>>>
>>>
>>> On 05/07/2012 18:09, Mark Hayes wrote:
>>>
>>> On Thu, Jul 5, 2012 at 9:44 AM, Peter Cameron
>>> <[EMAIL PROTECTED]>
>>> wrote:
>>>>
>>>> "This API is not recommended except as a stepping stone for systems that
>>>> currently uses Java interfaces to define RPC protocols. For new RPC
>>>> systems,
>>>> the specific API is preferred. For systems that process dynamic data,
>>>> the
>>>> generic API is probably best."
>>>>
>>>> What I'm confused by is the assertion that the generic API is "probably
>>>> best" for processing dynamic data.
>>>
>>>
>>> I am still fairly new to Avro but I think what the warning in the docs is
>>> trying to say is that the Specific API is better for static data, because
>>> reflection is slower.  If you're representing data using a Java bean,
>>> then
>>> your data is static (known at build time).
>>>
>>> --mark
>>>
>>>
>>>
>>> --
>>> Peter Cameron
>>> 2iC Limited
>>> T: +44 (0) 208 123 7479
>>> E: [EMAIL PROTECTED]
>>> W: www.2iCworld.com
>
>
>
> --
> Peter Cameron
> 2iC Limited
> T: +44 (0) 208 123 7479
> E: [EMAIL PROTECTED]
> W: www.2iCworld.com
>