Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> Partial lookup without full deserialization


Copy link to this message
-
Re: Partial lookup without full deserialization
You can specify a reader schema of simply {a:int}.  Avro will
efficiently skip missing fields when parsing values.  Note that you
still need the original, full schema (the "writer" schema).  This is
achieved through the schema resolution rules.

http://avro.apache.org/docs/current/spec.html#Schema+Resolution

Doug

On Thu, Oct 31, 2013 at 5:20 PM, Arvind Kalyan <[EMAIL PROTECTED]> wrote:
> Folks, say I serialize a GenericData.Record with some schema {a: int, b:
> string, c: array[int]} into a byte[] and send it over the wire.
>
> On the receiving side, once I have this byte[] is it possible for me to
> lookup just the field 'a' without incurring the cost of deserializing all
> the fields?
>
> Any other thoughts around trying to optimize partial lookups?
>
> thanks
> --
> Arvind Kalyan
> http://www.linkedin.com/in/base16
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB