Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro, mail # user - Is it possible to append to an already existing avro file


Copy link to this message
-
Re: Is it possible to append to an already existing avro file
Harsh J 2012-02-21, 04:29
Hi,

Use the appendTo feature of the DataFileWriter. See
http://avro.apache.org/docs/1.6.2/api/java/org/apache/avro/file/DataFileWriter.html#appendTo(java.io.File)

For a quick setup example, read also:
http://stackoverflow.com/questions/8806689/can-you-append-data-to-an-existing-avro-data-file

On Tue, Feb 21, 2012 at 3:15 AM, Vyacheslav Zholudev
<[EMAIL PROTECTED]> wrote:
> Hi,
>
> is it possible to append to an already existing avro file when it was
> written and closed before?
>
> If I use
> outputStream = fs.append(avroFilePath);
>
> then later on I get: java.io.IOException: Invalid sync!
>
> Probably because the schema is written twice and some other issues.
>
> If I useĀ outputStream = fs.create(avroFilePath); then the avro file gets
> overwritten.
>
> Thanks,
> Vyacheslav

--
Harsh J
Customer Ops. Engineer
Cloudera | http://tiny.cloudera.com/about