Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> Is DataFileWriter#append(datum) reentrant?


Copy link to this message
-
Is DataFileWriter#append(datum) reentrant?
Hi,

I am using AvroKeyRecordWriter, which wraps DataFileWriter#append(datum),
to create Avro.  As I write data, sometimes an error will occur due to an
encoding problem (e.g. a non-nullable field isn't set in a record).  I
would like to be able to log the AppendWriteException and continue writing
the remaining records.  Is this supported?  In other words, is
AvroKeyRecordWriter#write() guaranteed to be reentrant with respect to
AppendWriteException?

If not, is there a way to validate a datum against the schema before
writing?

Thanks,
Josh
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB