While porting Sergiy's task fail tests, I encountered a mismatch between
Java REEF and REEF.Net in the way task handler failures are handled.
Specifically, in REEF.Net an exception in the following task handlers will
crash the Evaluator:

* Close task
* Suspend task
* Task message

 

Within the code of TaskRuntime#Suspend the following comment exists:

// An Exception in suspend should crash the Evaluator.

Call suspend handler with no exception catch, which indeed crashes the
evaluator with no trace of the task exception i.e., the task exception is
lost.

 

Meanwhile, on the Java side (as evident from Sergiy's tests), it would seem
that the Evaluator does not crash on task handler exceptions. Looking into
the equivalent TaskRuntime (in Java), it would also seem that exceptions
raised in all task handlers only update the status of the task (to failed
with the exception information).

 

The question then is whose right? I'm going to run under the assumption that
Java has implemented the desired semantics and that it is the application's
responsibility to die if such exceptions are given to the driver.

 

If anyone disagrees then please speak up.

 

-Tyson
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB