Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro, mail # user - Avro Python appending data


Copy link to this message
-
Re: Avro Python appending data
Harsh J 2010-12-23, 04:16
Hi,

On Thu, Dec 23, 2010 at 6:29 AM, felix gao <[EMAIL PROTECTED]> wrote:
> Hi all,
>
> I am having trouble adding more data into a file.
>
> Environment: Python 2.6.5, avro-1.3.3-py2.6
>
> Program looks like this

I see you've read my blog post on Avro+Python :P
http://www.harshj.com/2010/04/25/writing-and-reading-avro-data-files-using-python/

> if I remove the second write_avro_file() call then everything is fine.  How
> to properly append more data into the file?

To append to an existing datafile, do not initialize the writer object
with a writers_schema again. Just create it using:

df_writer = datafile.DataFileWriter(
                    open(OUTFILE_NAME, 'wb'),
                    io.DatumWriter(),
                )

--
Harsh J
www.harshj.com