Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> Is it possible to append to an already existing avro file


+
Vyacheslav Zholudev 2012-02-20, 21:45
Copy link to this message
-
Re: Is it possible to append to an already existing avro file
Hi,

Use the appendTo feature of the DataFileWriter. See
http://avro.apache.org/docs/1.6.2/api/java/org/apache/avro/file/DataFileWriter.html#appendTo(java.io.File)

For a quick setup example, read also:
http://stackoverflow.com/questions/8806689/can-you-append-data-to-an-existing-avro-data-file

On Tue, Feb 21, 2012 at 3:15 AM, Vyacheslav Zholudev
<[EMAIL PROTECTED]> wrote:
> Hi,
>
> is it possible to append to an already existing avro file when it was
> written and closed before?
>
> If I use
> outputStream = fs.append(avroFilePath);
>
> then later on I get: java.io.IOException: Invalid sync!
>
> Probably because the schema is written twice and some other issues.
>
> If I useĀ outputStream = fs.create(avroFilePath); then the avro file gets
> overwritten.
>
> Thanks,
> Vyacheslav

--
Harsh J
Customer Ops. Engineer
Cloudera | http://tiny.cloudera.com/about
+
Vyacheslav Zholudev 2012-02-21, 15:29
+
Scott Carey 2012-02-21, 17:02
+
Vyacheslav Zholudev 2012-02-22, 09:57
+
Michael Malak 2013-02-01, 19:32
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB