-Re: Why ScannerTimeoutException is not handled in TableInputFormat?
Jan Lukavský 2012-03-12, 10:03
sorry I forgot to include the details of our environment. We are using
cdh3u3, which includes the patch (I think cdh3u2 did not). I went a bit
through the patches for the two issues (HBASE-4196) and (HBASE-4269) and
I think the problem is that the patch to HBASE-4196 did not change the
sematics for *mapreduce* package. The change was related to *mapred
*(the older API). We are using the new one and hence for us the
semantics was in fact changed by HBASE-4269. IMO, the correct way of
restoring the sematics would be to catch DNRIOEx only in the mapred
implementation of the record reader.
Shall I file another JIRA?
On 7.3.2012 00:25, Jonathan Hsieh wrote:
> Hi Jan,
> What version were you on before and what version are you on currently?
> The HBASE-4269 tried to restore the semantics changed in HBASE-4196 from
> 0.90.3 to 0.90.5. If we caught the exception we may skip records if the
> timeout exceptions are skipped which is undesirable.
> It might make sense to change Scanner Timeout to be derived from a
> different exception but this will take a bit of digging and testing to
> verify (DNRIOEx is used in about 40 places)
> On Fri, Feb 24, 2012 at 5:08 AM, Jan Lukavskï¿½
> <[EMAIL PROTECTED]>wrote:
>> Hi all,
>> patch to HBASE-4269 removed handling of ScannerTimeoutException from
>> TableRecordReader. Is there a reason for this? We are now seeing a lot of
>> ScannerTimeoutExceptions in our jobs, which were previously handled. The
>> problem is caused by ScannerTimeoutException being derived from
>> DoNotRetryIOException (why's that?). Would it be safe to just derive this
>> exception from IOException? And if not, should the TableRecordReader
>> explictly handle this exception? The same problem may arise with
>> LeaseException, which is also derived from DoNotRetryIOException. Both of
>> these exception may have the same cause and in my understanding can be
>> handled in the RecordReader without any harm. Or am I wrong?
>> Thanks for answer,