+
Ratner, Alan S 2012-11-21, 20:01
+
Mohammad Tariq 2012-11-21, 20:11
-
RE: Re: HBase Issues (perhaps related to 127.0.0.1)Ratner, Alan S 2012-11-21, 21:02
Thanks Mohammad. I set the clientPort but as I was already using the default value of 2181 it made no difference.
I cannot remove the 127.0.0.1 line from my hosts file. I connect to my servers via VPN from a Windows laptop using either NX or VNC and both apparently rely on the 127.0.0.1 IP address. This was not a problem with older versions of HBase (I used to use 0.20.x) so it seems to be something relatively new.
It seems I have a choice: access my servers remotely or run HBase and these 2 are mutually incompatible. I think my options are either:
a) revert to an old version of HBase
b) switch to Accumulo, or
c) switch to Cassandra.
Alan
-----Original Message-----
From: Mohammad Tariq [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, November 21, 2012 3:11 PM
To: [EMAIL PROTECTED]
Subject: EXT :Re: HBase Issues (perhaps related to 127.0.0.1)
Hello Alan,
It's better to keep 127.0.0.1 out of your /etc/hosts and make sure you
have proper DNS resolution as it plays an important role in proper Hbase
functioning. Also add the "hbase.zookeeper.property.clientPort" property in
your hbase-site.xml file and see if it works for you.
Regards,
Mohammad Tariq
On Thu, Nov 22, 2012 at 1:31 AM, Ratner, Alan S (IS) <[EMAIL PROTECTED]>wrote:
> I'd appreciate any suggestions as to how to get HBase up and running.
> Right now it dies after a few seconds on all servers. I am using Hadoop
> 1.0.4, ZooKeeper 3.4.4 and HBase 0.94.2 on Ubuntu.
>
> History: Yesterday I managed to get HBase 0.94.2 working but only after
> removing the 127.0.0.1 line from my /etc/hosts file (and synchronizing my
> clocks). All was fine until this morning when I realized I could not
> initiate remote log-ins to my servers (using VNC or NX) until I restored
> the 127.0.0.1 line in /etc/hosts. With that restored I am back to a
> non-working HBase.
>
> With HBase managing ZK I see the following in the HBase Master and ZK
> logs, respectively:
> 2012-11-21 13:40:22,236 WARN
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: Possibly transient
> ZooKeeper exception:
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase
>
> 2012-11-21 13:40:22,122 WARN org.apache.zookeeper.server.NIOServerCnxn:
> Exception causing close of session 0x0 due to java.io.IOException:
> ZooKeeperServer not running
>
> At roughly the same time (clocks not perfectly synchronized) I see this in
> a Regionserver log:
> 2012-11-21 13:40:57,727 WARN
> org.apache.zookeeper.client.ZooKeeperSaslClient: SecurityException:
> java.lang.SecurityException: Unable to locate a login configuration
> occurred when trying to find JAAS configuration.
> ...
> 2012-11-21 13:40:57,848 WARN
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: Possibly transient
> ZooKeeper exception:
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase/master
>
> Logs and configuration follows.
>
> Then I tried managing ZK myself and HBase then fails for seemingly
> different reasons.
> 2012-11-21 14:46:37,320 WARN
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: Node
> /hbase/backup-masters/hadoop1,60000,1353527196915 already deleted, and this
> is not a retry
>
> 2012-11-21 14:46:47,483 FATAL org.apache.hadoop.hbase.master.HMaster:
> Unhandled exception. Starting shutdown.
> java.net.ConnectException: Call to hadoop1/127.0.0.1:9000 failed on
> connection exception: java.net.ConnectException: Connection refused
>
> Both HMaster error logs (self-managed and me-managed ZK) mention the
> 127.0.0.1 IP address instead of referring to the server by its name
> (hadoop1) or its true IP address or simply as localhost.
>
> So, start-hbase.sh works OK (HB managing ZK):
> ngc@hadoop1:~/hbase-0.94.2$ bin/start-hbase.sh
> hadoop1: starting zookeeper, logging to
> /tmp/hbase-ngc/logs/hbase-ngc-zookeeper-hadoop1.out
> hadoop2: starting zookeeper, logging to
> /tmp/hbase-ngc/logs/hbase-ngc-zookeeper-hadoop2.out
I cannot remove the 127.0.0.1 line from my hosts file. I connect to my servers via VPN from a Windows laptop using either NX or VNC and both apparently rely on the 127.0.0.1 IP address. This was not a problem with older versions of HBase (I used to use 0.20.x) so it seems to be something relatively new.
It seems I have a choice: access my servers remotely or run HBase and these 2 are mutually incompatible. I think my options are either:
a) revert to an old version of HBase
b) switch to Accumulo, or
c) switch to Cassandra.
Alan
-----Original Message-----
From: Mohammad Tariq [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, November 21, 2012 3:11 PM
To: [EMAIL PROTECTED]
Subject: EXT :Re: HBase Issues (perhaps related to 127.0.0.1)
Hello Alan,
It's better to keep 127.0.0.1 out of your /etc/hosts and make sure you
have proper DNS resolution as it plays an important role in proper Hbase
functioning. Also add the "hbase.zookeeper.property.clientPort" property in
your hbase-site.xml file and see if it works for you.
Regards,
Mohammad Tariq
On Thu, Nov 22, 2012 at 1:31 AM, Ratner, Alan S (IS) <[EMAIL PROTECTED]>wrote:
> I'd appreciate any suggestions as to how to get HBase up and running.
> Right now it dies after a few seconds on all servers. I am using Hadoop
> 1.0.4, ZooKeeper 3.4.4 and HBase 0.94.2 on Ubuntu.
>
> History: Yesterday I managed to get HBase 0.94.2 working but only after
> removing the 127.0.0.1 line from my /etc/hosts file (and synchronizing my
> clocks). All was fine until this morning when I realized I could not
> initiate remote log-ins to my servers (using VNC or NX) until I restored
> the 127.0.0.1 line in /etc/hosts. With that restored I am back to a
> non-working HBase.
>
> With HBase managing ZK I see the following in the HBase Master and ZK
> logs, respectively:
> 2012-11-21 13:40:22,236 WARN
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: Possibly transient
> ZooKeeper exception:
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase
>
> 2012-11-21 13:40:22,122 WARN org.apache.zookeeper.server.NIOServerCnxn:
> Exception causing close of session 0x0 due to java.io.IOException:
> ZooKeeperServer not running
>
> At roughly the same time (clocks not perfectly synchronized) I see this in
> a Regionserver log:
> 2012-11-21 13:40:57,727 WARN
> org.apache.zookeeper.client.ZooKeeperSaslClient: SecurityException:
> java.lang.SecurityException: Unable to locate a login configuration
> occurred when trying to find JAAS configuration.
> ...
> 2012-11-21 13:40:57,848 WARN
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: Possibly transient
> ZooKeeper exception:
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase/master
>
> Logs and configuration follows.
>
> Then I tried managing ZK myself and HBase then fails for seemingly
> different reasons.
> 2012-11-21 14:46:37,320 WARN
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: Node
> /hbase/backup-masters/hadoop1,60000,1353527196915 already deleted, and this
> is not a retry
>
> 2012-11-21 14:46:47,483 FATAL org.apache.hadoop.hbase.master.HMaster:
> Unhandled exception. Starting shutdown.
> java.net.ConnectException: Call to hadoop1/127.0.0.1:9000 failed on
> connection exception: java.net.ConnectException: Connection refused
>
> Both HMaster error logs (self-managed and me-managed ZK) mention the
> 127.0.0.1 IP address instead of referring to the server by its name
> (hadoop1) or its true IP address or simply as localhost.
>
> So, start-hbase.sh works OK (HB managing ZK):
> ngc@hadoop1:~/hbase-0.94.2$ bin/start-hbase.sh
> hadoop1: starting zookeeper, logging to
> /tmp/hbase-ngc/logs/hbase-ngc-zookeeper-hadoop1.out
> hadoop2: starting zookeeper, logging to
> /tmp/hbase-ngc/logs/hbase-ngc-zookeeper-hadoop2.out
+
Michael Segel 2012-11-22, 00:40
