-Re: Scan exception when running MR
Bryan Beaudreault 2012-10-22, 16:22
I'm not on 0.94.1, but I've found a lot of situations that can cause
scanner timeouts and other scanner exceptions from M/R. The primary ones
probably still apply in later versions:
- Caching or batching set too high. If caching is set to, e.g. 1000,
and hbase.rpc.timeout is set to 30 seconds, it means you need to be able to
be able to process 1000 records in your mapper in under 30 seconds (minus
the overhead of actually returning that many records). Otherwise the
mapper's next call to next() will throw a timeout.
- Similar to the above, this can happen if just the logic in your mapper
is too heavy and taking too long. Just keep in mind that the
hbase.rpc.timeout can be triggered by the time between calls to next()
- hbase.rpc.timeout > hbase.regionserver.lease.period. If this is the
case, the RS will timeout first and kill the scan. Then your mapper will
call next() and since the scan no longer exists will throw a scan/lease
- The filters on the mapper scan are causing too many rows to be
skipped, such that not enough rows can be collected to return within the
Hope this helps and/or is still accurate for your version.
On Mon, Oct 22, 2012 at 11:51 AM, J Mohamed Zahoor <[EMAIL PROTECTED]> wrote:
> I am using 0.94.1
> On 22-Oct-2012, at 9:17 PM, J Mohamed Zahoor <[EMAIL PROTECTED]> wrote:
> > Hi
> > I am facing a scanner exception like this when i run a mr job.
> > Both the input and output are hbase tables (different tables)…
> > This comes sporadically on some mapper and all other mappers runs fine..
> > Even the failed mapper gets passed in the next attempt.
> > Any clue on what might be wrong?
> > java.lang.NullPointerException
> > at org.apache.hadoop.hbase.client.Scan.<init>(Scan.java:147)
> > at
> > at
> > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:731)
> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
> > at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at javax.security.auth.Subject.doAs(Subject.java:415)
> > at
> > at org.apache.hadoop.mapred.Child.main(Child.java:249)
> > ./zahoor