Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> Understanding responseTooSlow in HBase regionserver log


+
Mike Spreitzer 2012-04-26, 17:55
+
Jean-Daniel Cryans 2012-04-26, 18:31
Copy link to this message
-
Re: Understanding responseTooSlow in HBase regionserver log
Mike,
Check logResponse method
of org.apache.hadoop.hbase.ipc.WritableRpcEngine.Server.
This piece in particular :

 else if (params.length == 1 && instance instanceof HRegionServer &&
         params[0] instanceof Operation) {
       // annotate the response map with operation details
       responseInfo.putAll(((Operation) params[0]).toMap());
       // report to the log file
       LOG.warn("(*operation*" + tag + "): " +
           mapper.writeValueAsString(responseInfo));
     } else {
       // can't get JSON details, so just report call.toString() along with
       // a more generic tag.
       responseInfo.put("call", call.toString());
       LOG.warn("(*response*" + tag + "): " +
           mapper.writeValueAsString(responseInfo));
     }

On Thu, Apr 26, 2012 at 11:31 AM, Jean-Daniel Cryans <[EMAIL PROTECTED]>
wrote:
>
> You are right.
>
> FWIW I cannot find operationTooSlow in the source code, not sure
> what's going on there.
>
> J-D
>
> On Thu, Apr 26, 2012 at 10:55 AM, Mike Spreitzer <[EMAIL PROTECTED]>
wrote:
> > I found http://hbase.apache.org/book/ops.monitoring.html but am confused
> > about the distinction between operationTooSlow and responseTooSlow.  I
> > find the text "client operation" ambiguous, I am not sure whether that
> > means the side emitting this log entry is the client and the slow
> > operation happened on some server elsewhere or it is the other way
around
> > or something else.  I am running HBase-0.92.0 and found the following in
> > the log of a regionserver:
> >
> > 2012-04-26 13:17:30,698 WARN org.apache.hadoop.ipc.HBaseServer:
> > (responseTooSlow):
> > {"processingtimems":10340,"call":"execCoprocessor([B@f2d376,
> > <mycoprocessor/>, rpc version=1, client version=0,
methodsFingerPrint=0),
> > rpc version=1, client version=29,
> > methodsFingerPrint=54742778","client":"10.12.203.173:58531
","starttimems":1335460640355,"queuetimems":0,"class":"HRegionServer","responsesize":0,"method":"execCoprocessor"}
> >
> > 10.12.203.173 is not the IP address of the machine running this
> > regionserver, it is the IP address of the machine running my HBase
client
> > application.
> >
> > When is operationTooSlow used and when is responseTooSlow used?
> >
> > In my example above, am I correct that this was merely a warning issued
on
> > the regionserver saying that my coprocessor took a bit over 10 seconds
on
> > that regionserver and this did not cause any actual harm?  Note that my
> > HBase client application printed no complaints and appeared to work
> > correctly (although my evaluation of "correctness" was limited).
> >
> > Thanks,
> > Mike