Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro, mail # user - Random access in an avro file


Copy link to this message
-
Re: Random access in an avro file
Doug Cutting 2013-07-01, 17:51
On Mon, Jul 1, 2013 at 10:26 AM, [EMAIL PROTECTED]
<[EMAIL PROTECTED]> wrote:
> Out of curiosity, is maintaining sync markers while writing the file and
> then passing these markers to the readers while reading not a good way to
> achieve random access in avro?

Yes, seeking to the position of a sync marker is possible.  This is
what SortedKeyValueFile does.  You need to store the list of positions
of sync markers, and if seek is to a column value rather than a row
number, then you need to store these values (keys) with the positions.
 Those are what's in SortedKeyValueFile's "index" file.

Doug