Mike,

            Yes I do.

With this /etc/hosts HBase works but NX and VNC do not.

10.64.155.52 hadoop1.aj.c2fse.northgrum.com hadoop1 hbase-masterserver hbase-nameserver localhost

10.64.155.53 hadoop2.aj.c2fse.northgrum.com hadoop2 hbase-regionserver1

...

With this /etc/hosts NX and VNC work but HBase does not.

127.0.0.1 hadoop1 localhost.localdomain localhost

10.64.155.52 hadoop1.aj.c2fse.northgrum.com hadoop1 hbase-masterserver hbase-nameserver

10.64.155.53 hadoop2.aj.c2fse.northgrum.com hadoop2 hbase-regionserver1

...

I assume from your question that if I should try replacing

127.0.0.1 hadoop1 localhost.localdomain localhost

with simply:

127.0.0.1 localhost

Alan

-----Original Message-----
From: Michael Segel [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, November 21, 2012 7:40 PM
To: [EMAIL PROTECTED]
Subject: EXT :Re: HBase Issues (perhaps related to 127.0.0.1)

Hi,

Quick question...

DO you have 127.0.0.1 set to anything other than localhost?

If not, then it should be fine and you may want to revert to hard coded IP addresses on your other configuration files.

If you have Hadoop up and working, then you should be able to stand up HBase on top of that.

Just doing a quick look, and it seems that your name for your hadoop is resolving to your localhost.

What does your /etc/ hosts file look like?

How many machines in your cluster?

Have you thought about pulling down a 'free' copy of Cloudera, MapR or if Hortonworks has one ...

If you're thinking about using HBase as a standalone instance and don't care about Map/Reduce, maybe going with something else would make sense.

HTH

-Mike

On Nov 21, 2012, at 3:02 PM, "Ratner, Alan S (IS)" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:

> Thanks Mohammad.  I set the clientPort but as I was already using the default value of 2181 it made no difference.

>

> I cannot remove the 127.0.0.1 line from my hosts file.  I connect to my servers via VPN from a Windows laptop using either NX or VNC and both apparently rely on the 127.0.0.1 IP address.  This was not a problem with older versions of HBase (I used to use 0.20.x) so it seems to be something relatively new.

>

> It seems I have a choice: access my servers remotely or run HBase and these 2 are mutually incompatible.  I think my options are either:

> a) revert to an old version of HBase

> b) switch to Accumulo, or

> c) switch to Cassandra.

>

> Alan

>

>

> -----Original Message-----

> From: Mohammad Tariq [mailto:[EMAIL PROTECTED]]

> Sent: Wednesday, November 21, 2012 3:11 PM

> To: [EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>

> Subject: EXT :Re: HBase Issues (perhaps related to 127.0.0.1)

>

> Hello Alan,

>

>    It's better to keep 127.0.0.1 out of your /etc/hosts and make sure you

> have proper DNS resolution as it plays an important role in proper Hbase

> functioning. Also add the "hbase.zookeeper.property.clientPort" property in

> your hbase-site.xml file and see if it works for you.

>

> Regards,

>    Mohammad Tariq

>

>

>

> On Thu, Nov 22, 2012 at 1:31 AM, Ratner, Alan S (IS) <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>wrote:

>

>> I'd appreciate any suggestions as to how to get HBase up and running.

>> Right now it dies after a few seconds on all servers.  I am using Hadoop

>> 1.0.4, ZooKeeper 3.4.4 and HBase 0.94.2 on Ubuntu.

>>

>> History: Yesterday I managed to get HBase 0.94.2 working but only after

>> removing the 127.0.0.1 line from my /etc/hosts file (and synchronizing my

>> clocks).  All was fine until this morning when I realized I could not

>> initiate remote log-ins to my servers (using VNC or NX) until I restored

>> the 127.0.0.1 line in /etc/hosts.  With that restored I am back to a

>> non-working HBase.

>>

>> With HBase managing ZK I see the following in the HBase Master and ZK

>> logs, respectively:

>> 2012-11-21 13:40:22,236 WARN

>> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: Possibly transient

Hi Alan,

Yes. I am suggesting that.

Your 127.0.0.1 subnet should be localhost  only and then your other entries.
It looks like 10.64.155.52 is the external interface (eth0) for the machine hadoop1.

Adding it to 127.0.0.1 confuses HBase since it will use the first entry it sees. (Going from memory) So it will always look to local hosts.

I think that should fix your problem.

HTH

-Mike

On Nov 23, 2012, at 10:11 AM, "Ratner, Alan S (IS)" <[EMAIL PROTECTED]> wrote:

> Mike,
>
>
>
>            Yes I do.
>
>
>
> With this /etc/hosts HBase works but NX and VNC do not.
>
> 10.64.155.52 hadoop1.aj.c2fse.northgrum.com hadoop1 hbase-masterserver hbase-nameserver localhost
>
> 10.64.155.53 hadoop2.aj.c2fse.northgrum.com hadoop2 hbase-regionserver1
>
> ...
>
>
>
> With this /etc/hosts NX and VNC work but HBase does not.
>
> 127.0.0.1 hadoop1 localhost.localdomain localhost
>
> 10.64.155.52 hadoop1.aj.c2fse.northgrum.com hadoop1 hbase-masterserver hbase-nameserver
>
> 10.64.155.53 hadoop2.aj.c2fse.northgrum.com hadoop2 hbase-regionserver1
>
> ...
>
>
>
> I assume from your question that if I should try replacing
>
> 127.0.0.1 hadoop1 localhost.localdomain localhost
>
> with simply:
>
> 127.0.0.1 localhost
>
>
>
>
>
>
>
> Alan
>
>
>
>
>
> -----Original Message-----
> From: Michael Segel [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, November 21, 2012 7:40 PM
> To: [EMAIL PROTECTED]
> Subject: EXT :Re: HBase Issues (perhaps related to 127.0.0.1)
>
>
>
> Hi,
>
>
>
> Quick question...
>
>
>
> DO you have 127.0.0.1 set to anything other than localhost?
>
>
>
> If not, then it should be fine and you may want to revert to hard coded IP addresses on your other configuration files.
>
>
>
> If you have Hadoop up and working, then you should be able to stand up HBase on top of that.
>
>
>
> Just doing a quick look, and it seems that your name for your hadoop is resolving to your localhost.
>
> What does your /etc/ hosts file look like?
>
>
>
> How many machines in your cluster?
>
>
>
> Have you thought about pulling down a 'free' copy of Cloudera, MapR or if Hortonworks has one ...
>
>
>
> If you're thinking about using HBase as a standalone instance and don't care about Map/Reduce, maybe going with something else would make sense.
>
>
>
> HTH
>
>
>
> -Mike
>
>
>
> On Nov 21, 2012, at 3:02 PM, "Ratner, Alan S (IS)" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
>
>
>
>> Thanks Mohammad.  I set the clientPort but as I was already using the default value of 2181 it made no difference.
>
>>
>
>> I cannot remove the 127.0.0.1 line from my hosts file.  I connect to my servers via VPN from a Windows laptop using either NX or VNC and both apparently rely on the 127.0.0.1 IP address.  This was not a problem with older versions of HBase (I used to use 0.20.x) so it seems to be something relatively new.
>
>>
>
>> It seems I have a choice: access my servers remotely or run HBase and these 2 are mutually incompatible.  I think my options are either:
>
>> a) revert to an old version of HBase
>
>> b) switch to Accumulo, or
>
>> c) switch to Cassandra.
>
>>
>
>> Alan
>
>>
>
>>
>
>> -----Original Message-----
>
>> From: Mohammad Tariq [mailto:[EMAIL PROTECTED]]
>
>> Sent: Wednesday, November 21, 2012 3:11 PM
>
>> To: [EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>
>
>> Subject: EXT :Re: HBase Issues (perhaps related to 127.0.0.1)
>
>>
>
>> Hello Alan,
>
>>
>
>>   It's better to keep 127.0.0.1 out of your /etc/hosts and make sure you
>
>> have proper DNS resolution as it plays an important role in proper Hbase
>
>> functioning. Also add the "hbase.zookeeper.property.clientPort" property in
>
>> your hbase-site.xml file and see if it works for you.
>
>>
>
>> Regards,
>
>>   Mohammad Tariq
>
>>
>
>>
>
>>
>
>> On Thu, Nov 22, 2012 at 1:31 AM, Ratner, Alan S (IS) <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>wrote:
>

Hi,

I am also using Ubuntu 12.04, Zookeeper 3.4.4 HBase 0.94.2 and Hadoop 1.0.4. (64-bit nodes), I finally managed to have the HBase cluster up and running, below is the line in my /etc/hosts for your reference:

#127.0.0.1      localhost
127.0.0.1       localhost.localdomain localhost

According to my set up experience, below are my advices:
1) /etc/hosts: should not comment out 127.0.01 in /etc/hosts
2) Zookeeper: do not sync its "data" and "datalog" folders  to other Zookeeper servers in your deployment
3) check your start procedures:
  - check your firewall policies, make sure each server can use the required TCP/IP ports, especially port 2181 in your case
     - start Zookeeper first, need to make sure all other servers can access Zookeeper servers, use "/bin/zkCli.sh -server XXXX" or "echo ruok | nc XXXX 2181" to test all Zookeepers from each HBASE server.  
- start Hadoop, use JPS to make sure Namenode, SecondaryNameNode, Datanodes up and running, check LOG files of each servers
- start MapReduce if you need it
- start HBase, use JPS to check HBase's HMaster and HRegionServers, then wait a while use JPS to check HMaster and HRegionServers again, if them all HBASE servers gone but HADOOP still up and running,  most likely it would be HBASE configure issue in hbase-site.xml related to ZooKeeper settings or ZooKeeper configure/data issues.
Hope these help and good luck.
ac

Originally I have 7 nodes, 5 of them are 64-bit and 2 of them are 32-bit, all 64-bit servers are connected to network A and the two
On 24 Nov 2012, at 10:51 AM, Michael Segel wrote:

> Hi Alan,
>
> Yes. I am suggesting that.
>
> Your 127.0.0.1 subnet should be localhost  only and then your other entries.
> It looks like 10.64.155.52 is the external interface (eth0) for the machine hadoop1.
>
> Adding it to 127.0.0.1 confuses HBase since it will use the first entry it sees. (Going from memory) So it will always look to local hosts.
>
> I think that should fix your problem.
>
> HTH
>
> -Mike
>
> On Nov 23, 2012, at 10:11 AM, "Ratner, Alan S (IS)" <[EMAIL PROTECTED]> wrote:
>
>> Mike,
>>
>>
>>
>>           Yes I do.
>>
>>
>>
>> With this /etc/hosts HBase works but NX and VNC do not.
>>
>> 10.64.155.52 hadoop1.aj.c2fse.northgrum.com hadoop1 hbase-masterserver hbase-nameserver localhost
>>
>> 10.64.155.53 hadoop2.aj.c2fse.northgrum.com hadoop2 hbase-regionserver1
>>
>> ...
>>
>>
>>
>> With this /etc/hosts NX and VNC work but HBase does not.
>>
>> 127.0.0.1 hadoop1 localhost.localdomain localhost
>>
>> 10.64.155.52 hadoop1.aj.c2fse.northgrum.com hadoop1 hbase-masterserver hbase-nameserver
>>
>> 10.64.155.53 hadoop2.aj.c2fse.northgrum.com hadoop2 hbase-regionserver1
>>
>> ...
>>
>>
>>
>> I assume from your question that if I should try replacing
>>
>> 127.0.0.1 hadoop1 localhost.localdomain localhost
>>
>> with simply:
>>
>> 127.0.0.1 localhost
>>
>>
>>
>>
>>
>>
>>
>> Alan
>>
>>
>>
>>
>>
>> -----Original Message-----
>> From: Michael Segel [mailto:[EMAIL PROTECTED]]
>> Sent: Wednesday, November 21, 2012 7:40 PM
>> To: [EMAIL PROTECTED]
>> Subject: EXT :Re: HBase Issues (perhaps related to 127.0.0.1)
>>
>>
>>
>> Hi,
>>
>>
>>
>> Quick question...
>>
>>
>>
>> DO you have 127.0.0.1 set to anything other than localhost?
>>
>>
>>
>> If not, then it should be fine and you may want to revert to hard coded IP addresses on your other configuration files.
>>
>>
>>
>> If you have Hadoop up and working, then you should be able to stand up HBase on top of that.
>>
>>
>>
>> Just doing a quick look, and it seems that your name for your hadoop is resolving to your localhost.
>>
>> What does your /etc/ hosts file look like?
>>
>>
>>
>> How many machines in your cluster?
>>
>>
>>
>> Have you thought about pulling down a 'free' copy of Cloudera, MapR or if Hortonworks has one ...
>>
>>
>>
>> If you're thinking about using HBase as a standalone instance and don't care about Map/Reduce, maybe going with something else would make sense.

Thanks Mike - that solved the problem.
Alan Ratner, Northrop Grumman Information Systems

----- Original Message -----
From: Michael Segel [mailto:[EMAIL PROTECTED]]
Sent: Friday, November 23, 2012 08:51 PM
To: [EMAIL PROTECTED] <[EMAIL PROTECTED]>
Subject: Re: EXT :Re: HBase Issues (perhaps related to 127.0.0.1)

Hi Alan,

Yes. I am suggesting that.

Your 127.0.0.1 subnet should be localhost  only and then your other entries.
It looks like 10.64.155.52 is the external interface (eth0) for the machine hadoop1.

Adding it to 127.0.0.1 confuses HBase since it will use the first entry it sees. (Going from memory) So it will always look to local hosts.

I think that should fix your problem.

HTH

-Mike

On Nov 23, 2012, at 10:11 AM, "Ratner, Alan S (IS)" <[EMAIL PROTECTED]> wrote:

> Mike,
>
>
>
>            Yes I do.
>
>
>
> With this /etc/hosts HBase works but NX and VNC do not.
>
> 10.64.155.52 hadoop1.aj.c2fse.northgrum.com hadoop1 hbase-masterserver hbase-nameserver localhost
>
> 10.64.155.53 hadoop2.aj.c2fse.northgrum.com hadoop2 hbase-regionserver1
>
> ...
>
>
>
> With this /etc/hosts NX and VNC work but HBase does not.
>
> 127.0.0.1 hadoop1 localhost.localdomain localhost
>
> 10.64.155.52 hadoop1.aj.c2fse.northgrum.com hadoop1 hbase-masterserver hbase-nameserver
>
> 10.64.155.53 hadoop2.aj.c2fse.northgrum.com hadoop2 hbase-regionserver1
>
> ...
>
>
>
> I assume from your question that if I should try replacing
>
> 127.0.0.1 hadoop1 localhost.localdomain localhost
>
> with simply:
>
> 127.0.0.1 localhost
>
>
>
>
>
>
>
> Alan
>
>
>
>
>
> -----Original Message-----
> From: Michael Segel [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, November 21, 2012 7:40 PM
> To: [EMAIL PROTECTED]
> Subject: EXT :Re: HBase Issues (perhaps related to 127.0.0.1)
>
>
>
> Hi,
>
>
>
> Quick question...
>
>
>
> DO you have 127.0.0.1 set to anything other than localhost?
>
>
>
> If not, then it should be fine and you may want to revert to hard coded IP addresses on your other configuration files.
>
>
>
> If you have Hadoop up and working, then you should be able to stand up HBase on top of that.
>
>
>
> Just doing a quick look, and it seems that your name for your hadoop is resolving to your localhost.
>
> What does your /etc/ hosts file look like?
>
>
>
> How many machines in your cluster?
>
>
>
> Have you thought about pulling down a 'free' copy of Cloudera, MapR or if Hortonworks has one ...
>
>
>
> If you're thinking about using HBase as a standalone instance and don't care about Map/Reduce, maybe going with something else would make sense.
>
>
>
> HTH
>
>
>
> -Mike
>
>
>
> On Nov 21, 2012, at 3:02 PM, "Ratner, Alan S (IS)" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
>
>
>
>> Thanks Mohammad.  I set the clientPort but as I was already using the default value of 2181 it made no difference.
>
>>
>
>> I cannot remove the 127.0.0.1 line from my hosts file.  I connect to my servers via VPN from a Windows laptop using either NX or VNC and both apparently rely on the 127.0.0.1 IP address.  This was not a problem with older versions of HBase (I used to use 0.20.x) so it seems to be something relatively new.
>
>>
>
>> It seems I have a choice: access my servers remotely or run HBase and these 2 are mutually incompatible.  I think my options are either:
>
>> a) revert to an old version of HBase
>
>> b) switch to Accumulo, or
>
>> c) switch to Cassandra.
>
>>
>
>> Alan
>
>>
>
>>
>
>> -----Original Message-----
>
>> From: Mohammad Tariq [mailto:[EMAIL PROTECTED]]
>
>> Sent: Wednesday, November 21, 2012 3:11 PM
>
>> To: [EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>
>
>> Subject: EXT :Re: HBase Issues (perhaps related to 127.0.0.1)
>
>>
>
>> Hello Alan,
>
>>
>
>>   It's better to keep 127.0.0.1 out of your /etc/hosts and make sure you
>
>> have proper DNS resolution as it plays an important role in proper Hbase

On Sat, Nov 24, 2012 at 10:31 AM, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote:
> I am also using Ubuntu 12.04, Zookeeper 3.4.4 HBase 0.94.2 and Hadoop 1.0.4. (64-bit nodes), I finally managed to have the HBase cluster up and running, below is the line in my /etc/hosts for your reference:
>
> #127.0.0.1      localhost
> 127.0.0.1       localhost.localdomain localhost
>
> According to my set up experience, below are my advices:
> 1) /etc/hosts: should not comment out 127.0.01 in /etc/hosts
> 2) Zookeeper: do not sync its "data" and "datalog" folders  to other Zookeeper servers in your deployment
> 3) check your start procedures:
>         - check your firewall policies, make sure each server can use the required TCP/IP ports, especially port 2181 in your case
>         - start Zookeeper first, need to make sure all other servers can access Zookeeper servers, use "/bin/zkCli.sh -server XXXX" or "echo ruok | nc XXXX 2181" to test all Zookeepers from each HBASE server.
>         - start Hadoop, use JPS to make sure Namenode, SecondaryNameNode, Datanodes up and running, check LOG files of each servers
>         - start MapReduce if you need it
>         - start HBase, use JPS to check HBase's HMaster and HRegionServers, then wait a while use JPS to check HMaster and HRegionServers again, if them all HBASE servers gone but HADOOP still up and running,  most likely it would be HBASE configure issue in hbase-site.xml related to ZooKeeper settings or ZooKeeper configure/data issues.
>
>
> Hope these help and good luck.
> ac
>

Thanks ac for the clean instructions.  We have an ubuntu callout here
on localhost in /etc/hosts:
http://hbase.apache.org/book.html#basic.prerequisites  What else would
you suggest we add to the reference guide?

Thanks,
St.Ack
P.S. I used to have a friend named AC but in his case it stood for
"Anti-Christ".