Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> hmaster and regionserver died


Copy link to this message
-
RE: hmaster and regionserver died
Hi

Sorry if my reply mislead you.  I  meant to see the GC logs that should give
you an idea of if Full GC happened.

Regards
Ram

> -----Original Message-----
> From: Xiang Hua [mailto:[EMAIL PROTECTED]]
> Sent: Monday, October 15, 2012 12:42 PM
> To: [EMAIL PROTECTED]
> Subject: Re: hmaster and regionserver died
>
> We will check the zk log.
>
> On Monday, October 15, 2012, Ramkrishna.S.Vasudevan wrote:
>
> > Check your GC configurations.  Seems to that a Full GC has happened
> and the
> > Zookeeper thought that to be session expiry.
> >
> > Regards
> > Ram
> >
> > > -----Original Message-----
> > > From: Xiang Hua [mailto:[EMAIL PROTECTED]]
> > > Sent: Saturday, October 13, 2012 6:20 PM
> > > To: [EMAIL PROTECTED]
> > > Subject: hmaster and regionserver died
> > >
> > > Hi,
> > >    the HMaster died as well as regionservers, below is hmaster's
> log.
> > > could
> > > you please find what's problem?
> > >
> > >
> > > 2012-10-12 00:14:19,444 INFO org.apache.zookeeper.ClientCnxn:
> Socket
> > > connection established to bj-ecsxhm4f3I-r3-5-r810-2-hbase-stor-3/
> > > 10.20.16.34:2181, initiating session
> > > 2012-10-12 00:14:19,520 INFO org.apache.zookeeper.ClientCnxn:
> Session
> > > establishment complete on server bj-ecsxhm4f3I-r3-5-r810-2-hbase-
> stor-
> > > 3/
> > > 10.20.16.34:2181, sessionid = 0x139c539bc090002, negotiated timeout
> > > > 40000
> > > 2012-10-12 00:14:23,738 INFO org.apache.zookeeper.ClientCnxn:
> Client
> > > session timed out, have not heard from server in 15046ms for
> sessionid
> > > 0x239c539ba630001, closing socket connection and attempting
> reconnect
> > > 2012-10-12 00:14:24,246 INFO org.apache.zookeeper.ClientCnxn:
> Opening
> > > socket connection to server bj-ecsxhm4f3I-r3-5-r810-3-hbase-stor-2/
> > > 10.20.16.33:2181
> > > 2012-10-12 00:14:25,173 INFO org.apache.zookeeper.ClientCnxn:
> Client
> > > session timed out, have not heard from server in 15245ms for
> sessionid
> > > 0x139c539bc090003, closing socket connection and attempting
> reconnect
> > > 2012-10-12 00:14:25,328 INFO org.apache.zookeeper.ClientCnxn:
> Opening
> > > socket connection to server bj-ecsxhm4f3I-r3-5-r810-3-hbase-stor-2/
> > > 10.20.16.33:2181
> > > 2012-10-12 00:14:25,328 INFO org.apache.zookeeper.ClientCnxn:
> Socket
> > > connection established to bj-ecsxhm4f3I-r3-5-r810-3-hbase-stor-2/
> > > 10.20.16.33:2181, initiating session
> > > 2012-10-12 00:14:25,507 INFO org.apache.zookeeper.ClientCnxn:
> > > EventThread
> > > shut down
> > > 2012-10-12 00:14:25,507 INFO org.apache.zookeeper.ClientCnxn:
> Unable to
> > > reconnect to ZooKeeper service, session 0x139c539bc090003 has
> expired,
> > > closing socket connection
> > > 2012-10-12 00:14:27,247 INFO org.apache.zookeeper.ClientCnxn:
> Socket
> > > connection established to bj-ecsxhm4f3I-r3-5-r810-3-hbase-stor-2/
> > > 10.20.16.33:2181, initiating session
> > > 2012-10-12 00:14:27,248 WARN org.apache.zookeeper.ClientCnxn:
> Session
> > > 0x239c539ba630001 for server bj-ecsxhm4f3I-r3-5-r810-3-hbase-stor-
> 2/
> > > 10.20.16.33:2181, unexpected error, closing socket connection and
> > > attempting reconnect
> > > java.io.IOException: Connection reset by peer
> > >     at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
> > >     at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
> > >     at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:218)
> > >     at sun.nio.ch.IOUtil.read(IOUtil.java:186)
> > >     at
> sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:359)
> > >     at
> > >
> org.apache.zookeeper.ClientCnxn$SendThread.doIO(ClientCnxn.java:859)
> > >     at
> > >
> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1157)
> > > 2012-10-12 00:14:28,026 INFO org.apache.zookeeper.ClientCnxn:
> Opening
> > > socket connection to server bj-ecsxhm4f3I-r3-5-r810-2-hbase-stor-3/
> > > 10.20.16.34:2181
> > > 2012-10-12 00:14:41,359 INFO org.apache.zookeeper.ClientCnxn:
> Client
> > > session timed out, have not heard from server in 14007ms for
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB