Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> Random access in an avro file

Copy link to this message
Re: Random access in an avro file
On Mon, Jul 1, 2013 at 10:26 AM, [EMAIL PROTECTED]
> Out of curiosity, is maintaining sync markers while writing the file and
> then passing these markers to the readers while reading not a good way to
> achieve random access in avro?

Yes, seeking to the position of a sync marker is possible.  This is
what SortedKeyValueFile does.  You need to store the list of positions
of sync markers, and if seek is to a column value rather than a row
number, then you need to store these values (keys) with the positions.
 Those are what's in SortedKeyValueFile's "index" file.