Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> Random access in an avro file

kulkarni.swarnim@...) 2013-07-01, 15:37
Doug Cutting 2013-07-01, 17:05
kulkarni.swarnim@...) 2013-07-01, 17:26
Copy link to this message
Re: Random access in an avro file
On Mon, Jul 1, 2013 at 10:26 AM, [EMAIL PROTECTED]
> Out of curiosity, is maintaining sync markers while writing the file and
> then passing these markers to the readers while reading not a good way to
> achieve random access in avro?

Yes, seeking to the position of a sync marker is possible.  This is
what SortedKeyValueFile does.  You need to store the list of positions
of sync markers, and if seek is to a column value rather than a row
number, then you need to store these values (keys) with the positions.
 Those are what's in SortedKeyValueFile's "index" file.

kulkarni.swarnim@...) 2013-07-01, 18:50
kulkarni.swarnim@...) 2013-07-01, 22:22
Doug Cutting 2013-07-01, 22:52
Scott Carey 2013-07-02, 18:59