Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro, mail # user - Avro file - Seek to specific offset and read


Copy link to this message
-
Avro file - Seek to specific offset and read
Venkat 2013-02-20, 20:23
Hi All,
Using DataFileReader, I'm trying to read data from a specific [start-offset] to an [end-offset]. Both the start and end offsets are marked with synchronization markers using DataFileWriter.sync()

The following is the snipped I use to read the data back:

        DataFileReader<GenericRecord> fileReader = new DataFileReader<GenericRecord>(input, reader);     
        fileReader.seek(startOffset);  // set to the start-offset      

        while(fileReader.hasNext() && !fileReader.pastSync(endOffset))
        {
            GenericRecord gr = fileReader.next();
        }

This, however, reads & returns more records than what I wrote between the two offsets.
Appreciate your help regarding this.

Thanks