Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - HBASE -- Regionserver and QuorumPeer ?


Copy link to this message
-
Re: HBASE -- Regionserver and QuorumPeer ?
Suraj Varma 2012-07-03, 01:13
I think your devrackA-03 zookeeper is not quite "ok" - it doesn't seem
to be part of the quorum.

http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-JMX-Monitoring-suggestion-td6681354.html

>>> [hadoop@devrackA-00 ~]$ zookeeper-check
>>> devrackA-03
>>> imok
>>> This ZooKeeper instance is not currently serving requests
>>> This ZooKeeper instance is not currently serving requests

Check it's logs to see why it is not able to respond to stat command.
>>> imok[hadoop@devrackB-07 ~]$ echo stat | nc devrackA-03 2181
>>> This ZooKeeper instance is not currently serving requests
--Suraj

On Mon, Jul 2, 2012 at 5:51 PM, Jay Wilson
<[EMAIL PROTECTED]> wrote:
> When I do "locate hbase-site.xml", "locate hdfs-site.xml", and "locate
> core-site.xml" there are 2 locations for each on the HRegionServers.
> All files are either in $HADOOP_HOME/conf or $HBASE_HOME/conf and there
> are files of the same name in "example" directories.
>
> I moved my HRegionServers back to my original nodes that also have the
> HQuorumPeers on them.  My HMaster and HRegionServers are now running
> again.  I suspect they will terminate after 30 minutes like they have
> been doing, but at least they are running.
>
> ---
> Jay Wilson
>
> On 7/2/2012 4:43 PM, Suraj Varma wrote:
>> Ok - thanks for checking connectivity.
>>
>> I presume you already have doublechecked the hbase-site.xml in your
>> region server that points to the zookeeper and hdfs-site.xml pointed
>> to the namenode.
>>
>> I once got a similar error when HBase was picking up a stray
>> core-site.xml / hdfs-site.xml from the hdfs install or hbase-site.xml
>> from another hbase install (perhaps a stray local install)
>>
>> If connectivity is all right, and you are getting connection refused,
>> I think your region server is picking up the wrong configuration file.
>> So - do a "locate" on the region server configuration files to see if
>> there are others on the box.
>>
>> Just trying to eliminate basic setup issues ...
>> --Suraj
>>
>>
>> On Mon, Jul 2, 2012 at 3:55 PM, Jay Wilson
>> <[EMAIL PROTECTED]> wrote:
>>> First, thank you.
>>>
>>> I moved my HRegionservers not my HQuorumPeers.
>>>
>>> I have checked the network and everyone can talk to everyone.  I can
>>> even talk to my HQuorumPeers via "nc" from the nodes that should be
>>> running my HMaster on it and my HRegionservers.
>>>
>>> [hadoop@devrackA-00 ~]$ zookeeper-check
>>> devrackA-03
>>> imok
>>> This ZooKeeper instance is not currently serving requests
>>> This ZooKeeper instance is not currently serving requests
>>>
>>>
>>>
>>> devrackA-04
>>> imok
>>> Zookeeper version: 3.3.5-cdh3u4--1, built on 05/07/2012 20:10 GMT
>>> Clients:
>>>  /172.18.0.1:41582[0](queued=0,recved=1,sent=0)
>>>
>>> Latency min/avg/max: 0/0/0
>>> Received: 5
>>> Sent: 4
>>> Outstanding: 0
>>> Zxid: 0x0
>>> Mode: follower
>>> Node count: 4
>>>  /172.18.0.1:41583[0](queued=0,recved=1,sent=0)
>>>
>>>
>>>
>>>
>>> devrackA-05
>>> imok
>>> Zookeeper version: 3.3.5-cdh3u4--1, built on 05/07/2012 20:10 GMT
>>> Clients:
>>>  /172.18.0.1:35517[0](queued=0,recved=1,sent=0)
>>>
>>> Latency min/avg/max: 0/0/0
>>> Received: 5
>>> Sent: 4
>>> Outstanding: 0
>>> Zxid: 0x0
>>> Mode: follower
>>> Node count: 4
>>>  /172.18.0.1:35518[0](queued=0,recved=1,sent=0)
>>>
>>>
>>> ~~~~~~~~~~~~~~~~~~~~
>>>
>>>
>>> [hadoop@devrackA-06 ~]$ jps
>>> 21276 Jps
>>> 20641 DataNode
>>> [hadoop@devrackA-06 ~]$ echo ruok | nc devrackA-04 2181
>>> imok[hadoop@devrackA-06 ~]$ echo stat | nc devrackA-04 2181
>>> Zookeeper version: 3.3.5-cdh3u4--1, built on 05/07/2012 20:10 GMT
>>> Clients:
>>>  /172.18.0.7:37950[0](queued=0,recved=1,sent=0)
>>>
>>> Latency min/avg/max: 0/0/0
>>> Received: 8
>>> Sent: 7
>>> Outstanding: 0
>>> Zxid: 0x0
>>> Mode: follower
>>> Node count: 4
>>>
>>>
>>> ~~~~~~~~~~~~~~~~~~~
>>>
>>>
>>> [hadoop@devrackB-07 ~]$ echo ruok | nc devrackA-04 2181
>>> imok[hadoop@devrackB-07 ~]$ echo stat | nc devrackA-03 2181
>>> This ZooKeeper instance is not currently serving requests