Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> Partial lookup without full deserialization


Copy link to this message
-
Re: Partial lookup without full deserialization
You can specify a reader schema of simply {a:int}.  Avro will
efficiently skip missing fields when parsing values.  Note that you
still need the original, full schema (the "writer" schema).  This is
achieved through the schema resolution rules.

http://avro.apache.org/docs/current/spec.html#Schema+Resolution

Doug

On Thu, Oct 31, 2013 at 5:20 PM, Arvind Kalyan <[EMAIL PROTECTED]> wrote:
> Folks, say I serialize a GenericData.Record with some schema {a: int, b:
> string, c: array[int]} into a byte[] and send it over the wire.
>
> On the receiving side, once I have this byte[] is it possible for me to
> lookup just the field 'a' without incurring the cost of deserializing all
> the fields?
>
> Any other thoughts around trying to optimize partial lookups?
>
> thanks
> --
> Arvind Kalyan
> http://www.linkedin.com/in/base16
>