Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> Appending to .avro log files

Copy link to this message
Appending to .avro log files

I'm upgrading a logging program to append GenericRecords to a .avro file
instead of text (.tsv). I have a working schema that is used to convert
existing .tsv of the same format into .avro and that works fine.

When I run a test writing 30,000 bogus records, it runs but when I try
to use "avro-tools-1.7.3.jar tojson" on the output file, it reports:

"AvroRuntimeException: java.io.IOException: Invalid sync!"

The file is still open at this point since the logging program is
running. Is this expected behavior because it is still open? (getmeta
and getschema work fine).

I'm not sure if it has any bearing, since I never really understood the
function of the the AVRO sync interval; in this and the working programs
it is set to 1000000.

Any ideas appreciated.