Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro, mail # user - Random access in an avro file


+
kulkarni.swarnim@...) 2013-07-01, 15:37
+
Doug Cutting 2013-07-01, 17:05
+
kulkarni.swarnim@...) 2013-07-01, 17:26
Copy link to this message
-
Re: Random access in an avro file
Doug Cutting 2013-07-01, 17:51
On Mon, Jul 1, 2013 at 10:26 AM, [EMAIL PROTECTED]
<[EMAIL PROTECTED]> wrote:
> Out of curiosity, is maintaining sync markers while writing the file and
> then passing these markers to the readers while reading not a good way to
> achieve random access in avro?

Yes, seeking to the position of a sync marker is possible.  This is
what SortedKeyValueFile does.  You need to store the list of positions
of sync markers, and if seek is to a column value rather than a row
number, then you need to store these values (keys) with the positions.
 Those are what's in SortedKeyValueFile's "index" file.

Doug
+
kulkarni.swarnim@...) 2013-07-01, 18:50
+
kulkarni.swarnim@...) 2013-07-01, 22:22
+
Doug Cutting 2013-07-01, 22:52
+
Scott Carey 2013-07-02, 18:59