Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> Is DataFileWriter#append(datum) reentrant?


+
Josh Spiegel 2013-07-10, 14:37
Copy link to this message
-
Re: Is DataFileWriter#append(datum) reentrant?
The documentation for AppendWriteException says, "When this is thrown,
the file is unaltered and may continue to be appended to."  So, yes,
after you have caught this exception you may safely continue to append
entries to the still-open file.

Doug

On Wed, Jul 10, 2013 at 7:37 AM, Josh Spiegel <[EMAIL PROTECTED]> wrote:
> Hi,
>
> I am using AvroKeyRecordWriter, which wraps DataFileWriter#append(datum), to
> create Avro.  As I write data, sometimes an error will occur due to an
> encoding problem (e.g. a non-nullable field isn't set in a record).  I would
> like to be able to log the AppendWriteException and continue writing the
> remaining records.  Is this supported?  In other words, is
> AvroKeyRecordWriter#write() guaranteed to be reentrant with respect to
> AppendWriteException?
>
> If not, is there a way to validate a datum against the schema before
> writing?
>
> Thanks,
> Josh
+
Josh Spiegel 2013-07-10, 16:28
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB