Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> Avro Python appending data


Copy link to this message
-
Re: Avro Python appending data
Hi,

On Thu, Dec 23, 2010 at 6:29 AM, felix gao <[EMAIL PROTECTED]> wrote:
> Hi all,
>
> I am having trouble adding more data into a file.
>
> Environment: Python 2.6.5, avro-1.3.3-py2.6
>
> Program looks like this

I see you've read my blog post on Avro+Python :P
http://www.harshj.com/2010/04/25/writing-and-reading-avro-data-files-using-python/

> if I remove the second write_avro_file() call then everything is fine.  How
> to properly append more data into the file?

To append to an existing datafile, do not initialize the writer object
with a writers_schema again. Just create it using:

df_writer = datafile.DataFileWriter(
                    open(OUTFILE_NAME, 'wb'),
                    io.DatumWriter(),
                )

--
Harsh J
www.harshj.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB