Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> Avro file - Seek to specific offset and read


Copy link to this message
-
Avro file - Seek to specific offset and read
Hi All,
Using DataFileReader, I'm trying to read data from a specific [start-offset] to an [end-offset]. Both the start and end offsets are marked with synchronization markers using DataFileWriter.sync()

The following is the snipped I use to read the data back:

        DataFileReader<GenericRecord> fileReader = new DataFileReader<GenericRecord>(input, reader);     
        fileReader.seek(startOffset);  // set to the start-offset      

        while(fileReader.hasNext() && !fileReader.pastSync(endOffset))
        {
            GenericRecord gr = fileReader.next();
        }

This, however, reads & returns more records than what I wrote between the two offsets.
Appreciate your help regarding this.

Thanks
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB