Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> Partial lookup without full deserialization

Copy link to this message
Re: Partial lookup without full deserialization
You can specify a reader schema of simply {a:int}.  Avro will
efficiently skip missing fields when parsing values.  Note that you
still need the original, full schema (the "writer" schema).  This is
achieved through the schema resolution rules.



On Thu, Oct 31, 2013 at 5:20 PM, Arvind Kalyan <[EMAIL PROTECTED]> wrote:
> Folks, say I serialize a GenericData.Record with some schema {a: int, b:
> string, c: array[int]} into a byte[] and send it over the wire.
> On the receiving side, once I have this byte[] is it possible for me to
> lookup just the field 'a' without incurring the cost of deserializing all
> the fields?
> Any other thoughts around trying to optimize partial lookups?
> thanks
> --
> Arvind Kalyan
> http://www.linkedin.com/in/base16