Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # user - responseTooSlow
Mohit Anchlia
2012-12-21, 20:39
Mohammad Tariq
2012-12-21, 22:38
Mohammad Tariq
2012-12-21, 22:54
Mohit Anchlia
2012-12-21, 23:06
Mohammad Tariq
2012-12-21, 23:26
Mohit Anchlia
2012-12-21, 23:53
Mohammad Tariq
2012-12-22, 00:16
Mohammad Tariq
2012-12-22, 12:12
ramkrishna vasudevan
2012-12-22, 14:33
Mohit Anchlia
2012-12-22, 16:11
ramkrishna vasudevan
2012-12-22, 16:47
Azuryy Yu
2012-12-23, 01:50
+
Mohit Anchlia 2012-12-21, 20:39
+
Mohammad Tariq 2012-12-21, 22:38
+
Mohammad Tariq 2012-12-21, 22:54
+
Mohit Anchlia 2012-12-21, 23:06
+
Mohammad Tariq 2012-12-21, 23:26
+
Mohit Anchlia 2012-12-21, 23:53
+
Mohammad Tariq 2012-12-22, 00:16
Copy link to this message
-
Re: responseTooSlow
Mohit Anchlia 2012-12-22, 01:27
You mean batch multiple put?

On Fri, Dec 21, 2012 at 4:16 PM, Mohammad Tariq <[EMAIL PROTECTED]> wrote:

> It might be the RS which could not complete the operation in time. The
> appropriate way to find out is to monitor that RS's metrics and see if
> anything unusual is happening there. What type of keys are you using? It is
> time-series data?You might be a victim of RS hotspotting in that case or
> perhaps some other processes are eating up resources there. Try using
> "put(List<Put> puts)" instead of "put(Put put)" and see if it makes any
> difference.
>
> I'm afraid, I can't say anything with 100% confidence as there could be 'n'
> reasons which are not traceable from here. Some of the possible reasons
> could be :
> hotspotting region
> too much I/O wait due to Swapping
> overloaded disk
> slowness due to high cpu consumption
>
> Best Regards,
> Tariq
> +91-9741563634
> https://mtariq.jux.com/
>
>
> On Sat, Dec 22, 2012 at 5:23 AM, Mohit Anchlia <[EMAIL PROTECTED]
> >wrote:
>
> > I am just doing a put. This operation generally takes 10ms but in this
> case
> > it took more than 10sec. Nothing out of ordinary in the logs
> >
> > On Fri, Dec 21, 2012 at 3:26 PM, Mohammad Tariq <[EMAIL PROTECTED]>
> > wrote:
> >
> > > what exactly is the operation your trying to do?how is your network's
> > > health?is swapping too high at RS side?anything odd in your RS logs?
> > >
> > > Best Regards,
> > > Tariq
> > > +91-9741563634
> > > https://mtariq.jux.com/
> > >
> > >
> > > On Sat, Dec 22, 2012 at 4:36 AM, Mohit Anchlia <[EMAIL PROTECTED]
> > > >wrote:
> > >
> > > > I looked at that link, but couldn't find anything useful. How do I
> > check
> > > if
> > > > it was client who didn't write data within that time or if it was
> > region
> > > > server that didn't finish operation in time.
> > > >
> > > > On Fri, Dec 21, 2012 at 2:54 PM, Mohammad Tariq <[EMAIL PROTECTED]>
> > > > wrote:
> > > >
> > > > > The socket through which your client is communicating is getting
> > closed
> > > > > before the operation could get finished. May be it is taking longer
> > > than
> > > > > usual or something.
> > > > >
> > > > > Best Regards,
> > > > > Tariq
> > > > > +91-9741563634
> > > > > https://mtariq.jux.com/
> > > > >
> > > > >
> > > > >  On Sat, Dec 22, 2012 at 4:08 AM, Mohammad Tariq <
> [EMAIL PROTECTED]
> > >
> > > > > wrote:
> > > > >
> > > > > > Hello Mohit,
> > > > > >
> > > > > >       You might this link<
> > > > > http://hbase.apache.org/book/ops.monitoring.html>useful.
> > > > > >
> > > > > > Best Regards,
> > > > > > Tariq
> > > > > > +91-9741563634
> > > > > > https://mtariq.jux.com/
> > > > > >
> > > > > >
> > > > > > On Sat, Dec 22, 2012 at 2:09 AM, Mohit Anchlia <
> > > [EMAIL PROTECTED]
> > > > > >wrote:
> > > > > >
> > > > > >> Could someone help me understand what this really means. Is this
> > the
> > > > > >> network transfer taking long from client -> server or region
> > server
> > > > > taking
> > > > > >> long time writing to the memory?
> > > > > >>
> > > > > >> 2012-12-21 10:54:21,980 WARN org.apache.hadoop.ipc.HBaseServer:
> > > > > >> (responseTooSlow): {"processingtimems":135652,"call":"multi(
> > > > > >> org.apache.hadoop.hbase.client.MultiAction@28338472), rpc
> > > version=1,
> > > > > >> client
> > > > > >> version=29, methodsFingerPrint=54742778","client":"
> > 10.18.3.80:48218
> > > > > >>
> > > > > >>
> > > > >
> > > >
> > >
> >
> ","starttimems":1356115926326,"queuetimems":0,"class":"HRegionServer","responsesize":0,"method":"multi"}
> > > > > >> 2012-12-21 10:54:21,985 WARN org.apache.hadoop.ipc.HBaseServer:
> > IPC
> > > > > Server
> > > > > >> handler 26 on 60020 caught:
> > java.nio.channels.ClosedChannelException
> > > > > >>         at
> > > > > >>
> > > >
> > sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:133)
> > > > > >>         at
> > > > > sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:324)
> > > > > >>         at
> > > > > >>
> > >
+
Mohammad Tariq 2012-12-22, 12:12
+
ramkrishna vasudevan 2012-12-22, 14:33
+
Mohit Anchlia 2012-12-22, 16:11
+
ramkrishna vasudevan 2012-12-22, 16:47
+
Azuryy Yu 2012-12-23, 01:50