Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> 0.96.0 keep logging "Memstore is above high water mark"


Copy link to this message
-
Re: 0.96.0 keep logging "Memstore is above high water mark"
On Tue, Sep 3, 2013 at 10:30 AM, Kevin O'dell <[EMAIL PROTECTED]>wrote:

> Please don't remove the warn. It is important for troubleshooting and
> sizing.
>

There's nothing the WARN gives you that setting the "Unblocking updates..."
with the waiting time wouldn't give you, and you'd have a whole lot less
log spam.

Also, just to make sure I understand you, you say the warn is important for
troubleshooting, but have you used it? It's only in 0.95+.
> On Sep 3, 2013 1:29 PM, "Jean-Daniel Cryans" <[EMAIL PROTECTED]> wrote:
>
> > On Mon, Sep 2, 2013 at 12:04 PM, Jean-Marc Spaggiari <
> > [EMAIL PROTECTED]> wrote:
> >
> > > While running PE with 10 clients, server keep logging:
> > > 2013-09-02 14:56:59,919 WARN  [RpcServer.handler=5,port=44439]
> > > regionserver.MemStoreFlusher: Memstore is above high water mark and
> > > block 2362ms
> > > 2013-09-02 14:56:59,919 WARN  [RpcServer.handler=18,port=44439]
> > > regionserver.MemStoreFlusher: Memstore is above high water mark and
> > > block 2363ms
> > >
> >
> > Yeah that was added in HBASE-6466, it helps tracing when the handlers are
> > blocked on the memstores, else you have to match the "Blocking updates"
> > with the "Unblocking updates" lines. I'd just be fine adding the time
> into
> > the "Unblocking updates" line and remove that WARN.
> >
> >
> > >
> > > Then when test is over:
> > > 2013-09-02 14:57:02,280 WARN
> > > [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181] server.NIOServerCnxn:
> > > caught end of stream exception
> > > EndOfStreamException: Unable to read additional data from client
> > > sessionid 0x140dfdfc6270044, likely client has closed socket
> > >     at
> > > org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
> > >     at
> > >
> >
> org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
> > >     at java.lang.Thread.run(Thread.java:662)
> > >
> >
> > Your 10 clients are disconnecting from ZK, you're letting HBase manage
> it?
> >
> >
> > >
> > > /hbase-0.96.0$ cat logs/hbase-jmspaggiari-master-t430s.log  | grep
> > > "Memstore is above high water mark and block" | wc
> > >    3555   49770  514558
> > >
> > >
> > > /hbase-0.96.0$ cat logs/hbase-jmspaggiari-master-t430s.log  | grep
> > > "Unable to read additional data from client sessionid" | wc
> > >     102    1530   12852
> > >
> > > I guess it's only because of PE, but is this something which need to
> > > be looked at?
> > >
> > > JM
> > >
> >
>