Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> responseTooSlow


+
Mohit Anchlia 2012-12-21, 20:39
+
Mohammad Tariq 2012-12-21, 22:38
+
Mohammad Tariq 2012-12-21, 22:54
+
Mohit Anchlia 2012-12-21, 23:06
+
Mohammad Tariq 2012-12-21, 23:26
+
Mohit Anchlia 2012-12-21, 23:53
+
Mohammad Tariq 2012-12-22, 00:16
Copy link to this message
-
Re: responseTooSlow
You mean batch multiple put?

On Fri, Dec 21, 2012 at 4:16 PM, Mohammad Tariq <[EMAIL PROTECTED]> wrote:

> It might be the RS which could not complete the operation in time. The
> appropriate way to find out is to monitor that RS's metrics and see if
> anything unusual is happening there. What type of keys are you using? It is
> time-series data?You might be a victim of RS hotspotting in that case or
> perhaps some other processes are eating up resources there. Try using
> "put(List<Put> puts)" instead of "put(Put put)" and see if it makes any
> difference.
>
> I'm afraid, I can't say anything with 100% confidence as there could be 'n'
> reasons which are not traceable from here. Some of the possible reasons
> could be :
> hotspotting region
> too much I/O wait due to Swapping
> overloaded disk
> slowness due to high cpu consumption
>
> Best Regards,
> Tariq
> +91-9741563634
> https://mtariq.jux.com/
>
>
> On Sat, Dec 22, 2012 at 5:23 AM, Mohit Anchlia <[EMAIL PROTECTED]
> >wrote:
>
> > I am just doing a put. This operation generally takes 10ms but in this
> case
> > it took more than 10sec. Nothing out of ordinary in the logs
> >
> > On Fri, Dec 21, 2012 at 3:26 PM, Mohammad Tariq <[EMAIL PROTECTED]>
> > wrote:
> >
> > > what exactly is the operation your trying to do?how is your network's
> > > health?is swapping too high at RS side?anything odd in your RS logs?
> > >
> > > Best Regards,
> > > Tariq
> > > +91-9741563634
> > > https://mtariq.jux.com/
> > >
> > >
> > > On Sat, Dec 22, 2012 at 4:36 AM, Mohit Anchlia <[EMAIL PROTECTED]
> > > >wrote:
> > >
> > > > I looked at that link, but couldn't find anything useful. How do I
> > check
> > > if
> > > > it was client who didn't write data within that time or if it was
> > region
> > > > server that didn't finish operation in time.
> > > >
> > > > On Fri, Dec 21, 2012 at 2:54 PM, Mohammad Tariq <[EMAIL PROTECTED]>
> > > > wrote:
> > > >
> > > > > The socket through which your client is communicating is getting
> > closed
> > > > > before the operation could get finished. May be it is taking longer
> > > than
> > > > > usual or something.
> > > > >
> > > > > Best Regards,
> > > > > Tariq
> > > > > +91-9741563634
> > > > > https://mtariq.jux.com/
> > > > >
> > > > >
> > > > >  On Sat, Dec 22, 2012 at 4:08 AM, Mohammad Tariq <
> [EMAIL PROTECTED]
> > >
> > > > > wrote:
> > > > >
> > > > > > Hello Mohit,
> > > > > >
> > > > > >       You might this link<
> > > > > http://hbase.apache.org/book/ops.monitoring.html>useful.
> > > > > >
> > > > > > Best Regards,
> > > > > > Tariq
> > > > > > +91-9741563634
> > > > > > https://mtariq.jux.com/
> > > > > >
> > > > > >
> > > > > > On Sat, Dec 22, 2012 at 2:09 AM, Mohit Anchlia <
> > > [EMAIL PROTECTED]
> > > > > >wrote:
> > > > > >
> > > > > >> Could someone help me understand what this really means. Is this
> > the
> > > > > >> network transfer taking long from client -> server or region
> > server
> > > > > taking
> > > > > >> long time writing to the memory?
> > > > > >>
> > > > > >> 2012-12-21 10:54:21,980 WARN org.apache.hadoop.ipc.HBaseServer:
> > > > > >> (responseTooSlow): {"processingtimems":135652,"call":"multi(
> > > > > >> org.apache.hadoop.hbase.client.MultiAction@28338472), rpc
> > > version=1,
> > > > > >> client
> > > > > >> version=29, methodsFingerPrint=54742778","client":"
> > 10.18.3.80:48218
> > > > > >>
> > > > > >>
> > > > >
> > > >
> > >
> >
> ","starttimems":1356115926326,"queuetimems":0,"class":"HRegionServer","responsesize":0,"method":"multi"}
> > > > > >> 2012-12-21 10:54:21,985 WARN org.apache.hadoop.ipc.HBaseServer:
> > IPC
> > > > > Server
> > > > > >> handler 26 on 60020 caught:
> > java.nio.channels.ClosedChannelException
> > > > > >>         at
> > > > > >>
> > > >
> > sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:133)
> > > > > >>         at
> > > > > sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:324)
> > > > > >>         at
> > > > > >>
> > >
+
Mohammad Tariq 2012-12-22, 12:12
+
ramkrishna vasudevan 2012-12-22, 14:33
+
Mohit Anchlia 2012-12-22, 16:11
+
ramkrishna vasudevan 2012-12-22, 16:47
+
Azuryy Yu 2012-12-23, 01:50
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB