Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> HBASE -- Regionserver and QuorumPeer ?


Copy link to this message
-
Re: HBASE -- Regionserver and QuorumPeer ?
I think your devrackA-03 zookeeper is not quite "ok" - it doesn't seem
to be part of the quorum.

http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-JMX-Monitoring-suggestion-td6681354.html

>>> [hadoop@devrackA-00 ~]$ zookeeper-check
>>> devrackA-03
>>> imok
>>> This ZooKeeper instance is not currently serving requests
>>> This ZooKeeper instance is not currently serving requests

Check it's logs to see why it is not able to respond to stat command.
>>> imok[hadoop@devrackB-07 ~]$ echo stat | nc devrackA-03 2181
>>> This ZooKeeper instance is not currently serving requests
--Suraj

On Mon, Jul 2, 2012 at 5:51 PM, Jay Wilson
<[EMAIL PROTECTED]> wrote:
> When I do "locate hbase-site.xml", "locate hdfs-site.xml", and "locate
> core-site.xml" there are 2 locations for each on the HRegionServers.
> All files are either in $HADOOP_HOME/conf or $HBASE_HOME/conf and there
> are files of the same name in "example" directories.
>
> I moved my HRegionServers back to my original nodes that also have the
> HQuorumPeers on them.  My HMaster and HRegionServers are now running
> again.  I suspect they will terminate after 30 minutes like they have
> been doing, but at least they are running.
>
> ---
> Jay Wilson
>
> On 7/2/2012 4:43 PM, Suraj Varma wrote:
>> Ok - thanks for checking connectivity.
>>
>> I presume you already have doublechecked the hbase-site.xml in your
>> region server that points to the zookeeper and hdfs-site.xml pointed
>> to the namenode.
>>
>> I once got a similar error when HBase was picking up a stray
>> core-site.xml / hdfs-site.xml from the hdfs install or hbase-site.xml
>> from another hbase install (perhaps a stray local install)
>>
>> If connectivity is all right, and you are getting connection refused,
>> I think your region server is picking up the wrong configuration file.
>> So - do a "locate" on the region server configuration files to see if
>> there are others on the box.
>>
>> Just trying to eliminate basic setup issues ...
>> --Suraj
>>
>>
>> On Mon, Jul 2, 2012 at 3:55 PM, Jay Wilson
>> <[EMAIL PROTECTED]> wrote:
>>> First, thank you.
>>>
>>> I moved my HRegionservers not my HQuorumPeers.
>>>
>>> I have checked the network and everyone can talk to everyone.  I can
>>> even talk to my HQuorumPeers via "nc" from the nodes that should be
>>> running my HMaster on it and my HRegionservers.
>>>
>>> [hadoop@devrackA-00 ~]$ zookeeper-check
>>> devrackA-03
>>> imok
>>> This ZooKeeper instance is not currently serving requests
>>> This ZooKeeper instance is not currently serving requests
>>>
>>>
>>>
>>> devrackA-04
>>> imok
>>> Zookeeper version: 3.3.5-cdh3u4--1, built on 05/07/2012 20:10 GMT
>>> Clients:
>>>  /172.18.0.1:41582[0](queued=0,recved=1,sent=0)
>>>
>>> Latency min/avg/max: 0/0/0
>>> Received: 5
>>> Sent: 4
>>> Outstanding: 0
>>> Zxid: 0x0
>>> Mode: follower
>>> Node count: 4
>>>  /172.18.0.1:41583[0](queued=0,recved=1,sent=0)
>>>
>>>
>>>
>>>
>>> devrackA-05
>>> imok
>>> Zookeeper version: 3.3.5-cdh3u4--1, built on 05/07/2012 20:10 GMT
>>> Clients:
>>>  /172.18.0.1:35517[0](queued=0,recved=1,sent=0)
>>>
>>> Latency min/avg/max: 0/0/0
>>> Received: 5
>>> Sent: 4
>>> Outstanding: 0
>>> Zxid: 0x0
>>> Mode: follower
>>> Node count: 4
>>>  /172.18.0.1:35518[0](queued=0,recved=1,sent=0)
>>>
>>>
>>> ~~~~~~~~~~~~~~~~~~~~
>>>
>>>
>>> [hadoop@devrackA-06 ~]$ jps
>>> 21276 Jps
>>> 20641 DataNode
>>> [hadoop@devrackA-06 ~]$ echo ruok | nc devrackA-04 2181
>>> imok[hadoop@devrackA-06 ~]$ echo stat | nc devrackA-04 2181
>>> Zookeeper version: 3.3.5-cdh3u4--1, built on 05/07/2012 20:10 GMT
>>> Clients:
>>>  /172.18.0.7:37950[0](queued=0,recved=1,sent=0)
>>>
>>> Latency min/avg/max: 0/0/0
>>> Received: 8
>>> Sent: 7
>>> Outstanding: 0
>>> Zxid: 0x0
>>> Mode: follower
>>> Node count: 4
>>>
>>>
>>> ~~~~~~~~~~~~~~~~~~~
>>>
>>>
>>> [hadoop@devrackB-07 ~]$ echo ruok | nc devrackA-04 2181
>>> imok[hadoop@devrackB-07 ~]$ echo stat | nc devrackA-03 2181
>>> This ZooKeeper instance is not currently serving requests
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB