Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> Avro Python appending data


+
felix gao 2010-12-23, 00:59
+
Jeff Hammerbacher 2010-12-23, 03:13
Copy link to this message
-
Re: Avro Python appending data
Hi,

On Thu, Dec 23, 2010 at 6:29 AM, felix gao <[EMAIL PROTECTED]> wrote:
> Hi all,
>
> I am having trouble adding more data into a file.
>
> Environment: Python 2.6.5, avro-1.3.3-py2.6
>
> Program looks like this

I see you've read my blog post on Avro+Python :P
http://www.harshj.com/2010/04/25/writing-and-reading-avro-data-files-using-python/

> if I remove the second write_avro_file() call then everything is fine.  How
> to properly append more data into the file?

To append to an existing datafile, do not initialize the writer object
with a writers_schema again. Just create it using:

df_writer = datafile.DataFileWriter(
                    open(OUTFILE_NAME, 'wb'),
                    io.DatumWriter(),
                )

--
Harsh J
www.harshj.com
+
Harsh J 2010-12-23, 04:17
+
felix gao 2010-12-23, 17:30
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB