Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro, mail # user - Is DataFileWriter#append(datum) reentrant?


Copy link to this message
-
Is DataFileWriter#append(datum) reentrant?
Josh Spiegel 2013-07-10, 14:37
Hi,

I am using AvroKeyRecordWriter, which wraps DataFileWriter#append(datum),
to create Avro.  As I write data, sometimes an error will occur due to an
encoding problem (e.g. a non-nullable field isn't set in a record).  I
would like to be able to log the AppendWriteException and continue writing
the remaining records.  Is this supported?  In other words, is
AvroKeyRecordWriter#write() guaranteed to be reentrant with respect to
AppendWriteException?

If not, is there a way to validate a datum against the schema before
writing?

Thanks,
Josh
+
Doug Cutting 2013-07-10, 16:05
+
Josh Spiegel 2013-07-10, 16:28