Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> Random access in an avro file


+
kulkarni.swarnim@...) 2013-07-01, 15:37
+
Doug Cutting 2013-07-01, 17:05
+
kulkarni.swarnim@...) 2013-07-01, 17:26
Copy link to this message
-
Re: Random access in an avro file
On Mon, Jul 1, 2013 at 10:26 AM, [EMAIL PROTECTED]
<[EMAIL PROTECTED]> wrote:
> Out of curiosity, is maintaining sync markers while writing the file and
> then passing these markers to the readers while reading not a good way to
> achieve random access in avro?

Yes, seeking to the position of a sync marker is possible.  This is
what SortedKeyValueFile does.  You need to store the list of positions
of sync markers, and if seek is to a column value rather than a row
number, then you need to store these values (keys) with the positions.
 Those are what's in SortedKeyValueFile's "index" file.

Doug
+
kulkarni.swarnim@...) 2013-07-01, 18:50
+
kulkarni.swarnim@...) 2013-07-01, 22:22
+
Doug Cutting 2013-07-01, 22:52
+
Scott Carey 2013-07-02, 18:59
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB