why zookeeper bind 0.0.0.0 at miti-homed server?

$ netstat -lnp | grep 99
...
tcp        0      0 0.0.0.0:9900            0.0.0.0:*               LISTEN
   16536/java
tcp        0      0 0.0.0.0:9910            0.0.0.0:*               LISTEN
   16536/java
...

i deploy 3 zookeeper on the same server for test.
zoo.cfg
------------------------------
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=5
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=2
# the directory where the snapshot is stored.
dataDir=/data/zookeeper/data
# the port at which the clients will connect
clientPort=10000
clientPortAddress=172.27.31.48
server.1=172.27.31.48:9900:9910
server.2=172.27.31.48:9901:9911
server.3=172.27.31.48:9902:9912

reviewed class org.apache.zookeeper.server.quorum.QuorumCnxManager.Listener,
and I found this:

                    ss = ServerSocketChannel.open();
                    int port self.quorumPeers.get(self.getId()).electionAddr.getPort();
                    LOG.info("My election bind port: " + port);
                    ss.socket().setReuseAddress(true);
                    ss.socket().bind(new InetSocketAddress(port));
should I fire a bug report?

On Wed, Jun 15, 2011 at 11:33 AM, Isaac <[EMAIL PROTECTED]> wrote:

> why zookeeper bind 0.0.0.0 at miti-homed server?
>
> $ netstat -lnp | grep 99
> ...
> tcp        0      0 0.0.0.0:9900            0.0.0.0:*               LISTEN
>      16536/java
> tcp        0      0 0.0.0.0:9910            0.0.0.0:*               LISTEN
>      16536/java
> ...
>
> i deploy 3 zookeeper on the same server for test.
> zoo.cfg
> ------------------------------
> # The number of milliseconds of each tick
> tickTime=2000
> # The number of ticks that the initial
> # synchronization phase can take
> initLimit=5
> # The number of ticks that can pass between
> # sending a request and getting an acknowledgement
> syncLimit=2
> # the directory where the snapshot is stored.
> dataDir=/data/zookeeper/data
> # the port at which the clients will connect
> clientPort=10000
> clientPortAddress=172.27.31.48
> server.1=172.27.31.48:9900:9910
> server.2=172.27.31.48:9901:9911
> server.3=172.27.31.48:9902:9912
>
>

and more invocations of new InetSocketAddress(int port);
[image: image.png]

On Wed, Jun 15, 2011 at 1:24 PM, Isaac <[EMAIL PROTECTED]> wrote:

> reviewed class
> org.apache.zookeeper.server.quorum.QuorumCnxManager.Listener, and I found
> this:
>
>                     ss = ServerSocketChannel.open();
>                     int port > self.quorumPeers.get(self.getId()).electionAddr.getPort();
>                     LOG.info("My election bind port: " + port);
>                     ss.socket().setReuseAddress(true);
>                     ss.socket().bind(new InetSocketAddress(port));
> should I fire a bug report?
>
> On Wed, Jun 15, 2011 at 11:33 AM, Isaac <[EMAIL PROTECTED]> wrote:
>
>> why zookeeper bind 0.0.0.0 at miti-homed server?
>>
>> $ netstat -lnp | grep 99
>> ...
>> tcp        0      0 0.0.0.0:9900            0.0.0.0:*
>> LISTEN      16536/java
>> tcp        0      0 0.0.0.0:9910            0.0.0.0:*
>> LISTEN      16536/java
>> ...
>>
>> i deploy 3 zookeeper on the same server for test.
>> zoo.cfg
>> ------------------------------
>> # The number of milliseconds of each tick
>> tickTime=2000
>> # The number of ticks that the initial
>> # synchronization phase can take
>> initLimit=5
>> # The number of ticks that can pass between
>> # sending a request and getting an acknowledgement
>> syncLimit=2
>> # the directory where the snapshot is stored.
>> dataDir=/data/zookeeper/data
>> # the port at which the clients will connect
>> clientPort=10000
>> clientPortAddress=172.27.31.48
>> server.1=172.27.31.48:9900:9910
>> server.2=172.27.31.48:9901:9911
>> server.3=172.27.31.48:9902:9912
>>
>>
>

So that other machines can contact it.

Seriously, why did you have an expectation it would not bind to 0.0.0.0?

The configuration file tells how to reach OTHER servers.  It doesn't
limit which interfaces to use for incoming connections.

On Wed, Jun 15, 2011 at 5:33 AM, Isaac <[EMAIL PROTECTED]> wrote:
> why zookeeper bind 0.0.0.0 at miti-homed server?
>
> $ netstat -lnp | grep 99
> ...
> tcp        0      0 0.0.0.0:9900 ��          0.0.0.0:*               LISTEN
>   16536/java
> tcp        0      0 0.0.0.0:9910 ��          0.0.0.0:*               LISTEN
>   16536/java
> ...
>
> i deploy 3 zookeeper on the same server for test.
> zoo.cfg
> ------------------------------
> # The number of milliseconds of each tick
> tickTime=2000
> # The number of ticks that the initial
> # synchronization phase can take
> initLimit=5
> # The number of ticks that can pass between
> # sending a request and getting an acknowledgement
> syncLimit=2
> # the directory where the snapshot is stored.
> dataDir=/data/zookeeper/data
> # the port at which the clients will connect
> clientPort=10000
> clientPortAddress=172.27.31.48
> server.1=172.27.31.48:9900:9910
> server.2=172.27.31.48:9901:9911
> server.3=172.27.31.48:9902:9912
>

just company rule,for security reason.

e.g. 172.27.* for intranet communication, 8.8.* from this interface can
reach Internet.

i know iptables and other firewall rules can avoid this, but rule is rule.

I think others maybe encounter this problem, that's why clientPortAddress is
added in 3.3.0.

On Wed, Jun 15, 2011 at 3:53 PM, Ted Dunning <[EMAIL PROTECTED]> wrote:

> So that other machines can contact it.
>
> Seriously, why did you have an expectation it would not bind to 0.0.0.0?
>
> The configuration file tells how to reach OTHER servers.  It doesn't
> limit which interfaces to use for incoming connections.
>
> On Wed, Jun 15, 2011 at 5:33 AM, Isaac <[EMAIL PROTECTED]> wrote:
> > why zookeeper bind 0.0.0.0 at miti-homed server?
> >
> > $ netstat -lnp | grep 99
> > ...
> > tcp        0      0 0.0.0.0:9900            0.0.0.0:*
> LISTEN
> >   16536/java
> > tcp        0      0 0.0.0.0:9910            0.0.0.0:*
> LISTEN
> >   16536/java
> > ...
> >
> > i deploy 3 zookeeper on the same server for test.
> > zoo.cfg
> > ------------------------------
> > # The number of milliseconds of each tick
> > tickTime=2000
> > # The number of ticks that the initial
> > # synchronization phase can take
> > initLimit=5
> > # The number of ticks that can pass between
> > # sending a request and getting an acknowledgement
> > syncLimit=2
> > # the directory where the snapshot is stored.
> > dataDir=/data/zookeeper/data
> > # the port at which the clients will connect
> > clientPort=10000
> > clientPortAddress=172.27.31.48
> > server.1=172.27.31.48:9900:9910
> > server.2=172.27.31.48:9901:9911
> > server.3=172.27.31.48:9902:9912
> >
>

On Wed, Jun 15, 2011 at 3:14 AM, Isaac <[EMAIL PROTECTED]> wrote:
> just company rule,for security reason.
>
> e.g. 172.27.* for intranet communication, 8.8.* from this interface can
> reach Internet.
>
> i know iptables and other firewall rules can avoid this, but rule is rule.
>
> I think others maybe encounter this problem, that's why clientPortAddress is
> added in 3.3.0.

Yes, that's definitely the case. (I added it :-) ). FWIW, some
background: the typical deployment scenario early in the life of ZK
was inside a large datacenter, where everything had multiple levels of
firewalls. There was never any concern about the binding behavior
because ZK was put behind one of these firewalls, and only the client
port was exposed. As the deployment scenarios have become more diverse
(ec2 for example), there's been more interest in this type of
functionality. Thanks for picking this up! Appreciated.

Regards,

Patrick

Obviously the server.* settings are for connecting to other servers, I
believe he is referring to the clientPort and clientPortAddress combination.
These looks suspiciously like they are for choosing an IP and Port combo to
bind on..
Thanks,
Kiall
On Wed, Jun 15, 2011 at 8:53 AM, Ted Dunning <[EMAIL PROTECTED]> wrote:

> So that other machines can contact it.
>
> Seriously, why did you have an expectation it would not bind to 0.0.0.0?
>
> The configuration file tells how to reach OTHER servers.  It doesn't
> limit which interfaces to use for incoming connections.
>
> On Wed, Jun 15, 2011 at 5:33 AM, Isaac <[EMAIL PROTECTED]> wrote:
> > why zookeeper bind 0.0.0.0 at miti-homed server?
> >
> > $ netstat -lnp | grep 99
> > ...
> > tcp        0      0 0.0.0.0:9900            0.0.0.0:*
> LISTEN
> >   16536/java
> > tcp        0      0 0.0.0.0:9910            0.0.0.0:*
> LISTEN
> >   16536/java
> > ...
> >
> > i deploy 3 zookeeper on the same server for test.
> > zoo.cfg
> > ------------------------------
> > # The number of milliseconds of each tick
> > tickTime=2000
> > # The number of ticks that the initial
> > # synchronization phase can take
> > initLimit=5
> > # The number of ticks that can pass between
> > # sending a request and getting an acknowledgement
> > syncLimit=2
> > # the directory where the snapshot is stored.
> > dataDir=/data/zookeeper/data
> > # the port at which the clients will connect
> > clientPort=10000
> > clientPortAddress=172.27.31.48
> > server.1=172.27.31.48:9900:9910
> > server.2=172.27.31.48:9901:9911
> > server.3=172.27.31.48:9902:9912
> >
>

I suggested something similar a while back.  I too don't see a reason to
bind to all interfaces when we know the interface other servers are going to
contact us on.  See this thread, where I propose a patch.  If there's
interest, I can attach this patch to a JIRA you open.  I've been using this
for about a year now.

http://mail-archives.apache.org/mod_mbox/hadoop-zookeeper-dev/201008.mbox/%3CAANLkTikkT97Djqt3CU=H2+[EMAIL PROTECTED]%3E

~Jared

On Wed, Jun 15, 2011 at 6:52 AM, Kiall Mac Innes <[EMAIL PROTECTED]> wrote:

> Obviously the server.* settings are for connecting to other servers, I
> believe he is referring to the clientPort and clientPortAddress
> combination.
> These looks suspiciously like they are for choosing an IP and Port combo to
> bind on..
>
>
> Thanks,
> Kiall
>
>
> On Wed, Jun 15, 2011 at 8:53 AM, Ted Dunning <[EMAIL PROTECTED]>
> wrote:
>
> > So that other machines can contact it.
> >
> > Seriously, why did you have an expectation it would not bind to 0.0.0.0?
> >
> > The configuration file tells how to reach OTHER servers.  It doesn't
> > limit which interfaces to use for incoming connections.
> >
> > On Wed, Jun 15, 2011 at 5:33 AM, Isaac <[EMAIL PROTECTED]> wrote:
> > > why zookeeper bind 0.0.0.0 at miti-homed server?
> > >
> > > $ netstat -lnp | grep 99
> > > ...
> > > tcp        0      0 0.0.0.0:9900            0.0.0.0:*
> > LISTEN
> > >   16536/java
> > > tcp        0      0 0.0.0.0:9910            0.0.0.0:*
> > LISTEN
> > >   16536/java
> > > ...
> > >
> > > i deploy 3 zookeeper on the same server for test.
> > > zoo.cfg
> > > ------------------------------
> > > # The number of milliseconds of each tick
> > > tickTime=2000
> > > # The number of ticks that the initial
> > > # synchronization phase can take
> > > initLimit=5
> > > # The number of ticks that can pass between
> > > # sending a request and getting an acknowledgement
> > > syncLimit=2
> > > # the directory where the snapshot is stored.
> > > dataDir=/data/zookeeper/data
> > > # the port at which the clients will connect
> > > clientPort=10000
> > > clientPortAddress=172.27.31.48
> > > server.1=172.27.31.48:9900:9910
> > > server.2=172.27.31.48:9901:9911
> > > server.3=172.27.31.48:9902:9912
> > >
> >
>

Jared,

Sounds like you have a quorum of users interested in this patch.

On Wed, Jun 15, 2011 at 2:55 PM, Jared Cantwell <[EMAIL PROTECTED]>wrote:

> I suggested something similar a while back.  I too don't see a reason to
> bind to all interfaces when we know the interface other servers are going
> to
> contact us on.  See this thread, where I propose a patch.  If there's
> interest, I can attach this patch to a JIRA you open.  I've been using this
> for about a year now.
>
>
> http://mail-archives.apache.org/mod_mbox/hadoop-zookeeper-dev/201008.mbox/%3CAANLkTikkT97Djqt3CU=H2+[EMAIL PROTECTED]%3E
>
> ~Jared
>
> On Wed, Jun 15, 2011 at 6:52 AM, Kiall Mac Innes <[EMAIL PROTECTED]>
> wrote:
>
> > Obviously the server.* settings are for connecting to other servers, I
> > believe he is referring to the clientPort and clientPortAddress
> > combination.
> > These looks suspiciously like they are for choosing an IP and Port combo
> to
> > bind on..
> >
> >
> > Thanks,
> > Kiall
> >
> >
> > On Wed, Jun 15, 2011 at 8:53 AM, Ted Dunning <[EMAIL PROTECTED]>
> > wrote:
> >
> > > So that other machines can contact it.
> > >
> > > Seriously, why did you have an expectation it would not bind to
> 0.0.0.0?
> > >
> > > The configuration file tells how to reach OTHER servers.  It doesn't
> > > limit which interfaces to use for incoming connections.
> > >
> > > On Wed, Jun 15, 2011 at 5:33 AM, Isaac <[EMAIL PROTECTED]> wrote:
> > > > why zookeeper bind 0.0.0.0 at miti-homed server?
> > > >
> > > > $ netstat -lnp | grep 99
> > > > ...
> > > > tcp        0      0 0.0.0.0:9900            0.0.0.0:*
> > > LISTEN
> > > >   16536/java
> > > > tcp        0      0 0.0.0.0:9910            0.0.0.0:*
> > > LISTEN
> > > >   16536/java
> > > > ...
> > > >
> > > > i deploy 3 zookeeper on the same server for test.
> > > > zoo.cfg
> > > > ------------------------------
> > > > # The number of milliseconds of each tick
> > > > tickTime=2000
> > > > # The number of ticks that the initial
> > > > # synchronization phase can take
> > > > initLimit=5
> > > > # The number of ticks that can pass between
> > > > # sending a request and getting an acknowledgement
> > > > syncLimit=2
> > > > # the directory where the snapshot is stored.
> > > > dataDir=/data/zookeeper/data
> > > > # the port at which the clients will connect
> > > > clientPort=10000
> > > > clientPortAddress=172.27.31.48
> > > > server.1=172.27.31.48:9900:9910
> > > > server.2=172.27.31.48:9901:9911
> > > > server.3=172.27.31.48:9902:9912
> > > >
> > >
> >
>

I created a JIRA for this and will attach my patch to it soon.  I think it
might need minor cleanup to actually be submitted, but its a start.  Isaac,
perhaps you can push the patch to get it committed?

https://issues.apache.org/jira/browse/ZOOKEEPER-1096

~Jared

On Wed, Jun 15, 2011 at 12:47 PM, Ted Dunning <[EMAIL PROTECTED]> wrote:

> Jared,
>
> Sounds like you have a quorum of users interested in this patch.
>
> On Wed, Jun 15, 2011 at 2:55 PM, Jared Cantwell <[EMAIL PROTECTED]
> >wrote:
>
> > I suggested something similar a while back.  I too don't see a reason to
> > bind to all interfaces when we know the interface other servers are going
> > to
> > contact us on.  See this thread, where I propose a patch.  If there's
> > interest, I can attach this patch to a JIRA you open.  I've been using
> this
> > for about a year now.
> >
> >
> >
> http://mail-archives.apache.org/mod_mbox/hadoop-zookeeper-dev/201008.mbox/%3CAANLkTikkT97Djqt3CU=H2+[EMAIL PROTECTED]%3E
> >
> > ~Jared
> >
> > On Wed, Jun 15, 2011 at 6:52 AM, Kiall Mac Innes <[EMAIL PROTECTED]>
> > wrote:
> >
> > > Obviously the server.* settings are for connecting to other servers, I
> > > believe he is referring to the clientPort and clientPortAddress
> > > combination.
> > > These looks suspiciously like they are for choosing an IP and Port
> combo
> > to
> > > bind on..
> > >
> > >
> > > Thanks,
> > > Kiall
> > >
> > >
> > > On Wed, Jun 15, 2011 at 8:53 AM, Ted Dunning <[EMAIL PROTECTED]>
> > > wrote:
> > >
> > > > So that other machines can contact it.
> > > >
> > > > Seriously, why did you have an expectation it would not bind to
> > 0.0.0.0?
> > > >
> > > > The configuration file tells how to reach OTHER servers.  It doesn't
> > > > limit which interfaces to use for incoming connections.
> > > >
> > > > On Wed, Jun 15, 2011 at 5:33 AM, Isaac <[EMAIL PROTECTED]> wrote:
> > > > > why zookeeper bind 0.0.0.0 at miti-homed server?
> > > > >
> > > > > $ netstat -lnp | grep 99
> > > > > ...
> > > > > tcp        0      0 0.0.0.0:9900            0.0.0.0:*
> > > > LISTEN
> > > > >   16536/java
> > > > > tcp        0      0 0.0.0.0:9910            0.0.0.0:*
> > > > LISTEN
> > > > >   16536/java
> > > > > ...
> > > > >
> > > > > i deploy 3 zookeeper on the same server for test.
> > > > > zoo.cfg
> > > > > ------------------------------
> > > > > # The number of milliseconds of each tick
> > > > > tickTime=2000
> > > > > # The number of ticks that the initial
> > > > > # synchronization phase can take
> > > > > initLimit=5
> > > > > # The number of ticks that can pass between
> > > > > # sending a request and getting an acknowledgement
> > > > > syncLimit=2
> > > > > # the directory where the snapshot is stored.
> > > > > dataDir=/data/zookeeper/data
> > > > > # the port at which the clients will connect
> > > > > clientPort=10000
> > > > > clientPortAddress=172.27.31.48
> > > > > server.1=172.27.31.48:9900:9910
> > > > > server.2=172.27.31.48:9901:9911
> > > > > server.3=172.27.31.48:9902:9912
> > > > >
> > > >
> > >
> >
>

zookeeper finds its id in /${dataDir}/myid, and the look for its own
host,and port configuration in server.*  and clientPort,clientPortAddress is
also used.

please notice that zookeeper listening on 3 different port.

On Wed, Jun 15, 2011 at 8:52 PM, Kiall Mac Innes <[EMAIL PROTECTED]> wrote:

> Obviously the server.* settings are for connecting to other servers, I
> believe he is referring to the clientPort and clientPortAddress
> combination.
> These looks suspiciously like they are for choosing an IP and Port combo to
> bind on..
>
>
> Thanks,
> Kiall
>
>
> On Wed, Jun 15, 2011 at 8:53 AM, Ted Dunning <[EMAIL PROTECTED]>
> wrote:
>
> > So that other machines can contact it.
> >
> > Seriously, why did you have an expectation it would not bind to 0.0.0.0?
> >
> > The configuration file tells how to reach OTHER servers.  It doesn't
> > limit which interfaces to use for incoming connections.
> >
> > On Wed, Jun 15, 2011 at 5:33 AM, Isaac <[EMAIL PROTECTED]> wrote:
> > > why zookeeper bind 0.0.0.0 at miti-homed server?
> > >
> > > $ netstat -lnp | grep 99
> > > ...
> > > tcp        0      0 0.0.0.0:9900            0.0.0.0:*
> > LISTEN
> > >   16536/java
> > > tcp        0      0 0.0.0.0:9910            0.0.0.0:*
> > LISTEN
> > >   16536/java
> > > ...
> > >
> > > i deploy 3 zookeeper on the same server for test.
> > > zoo.cfg
> > > ------------------------------
> > > # The number of milliseconds of each tick
> > > tickTime=2000
> > > # The number of ticks that the initial
> > > # synchronization phase can take
> > > initLimit=5
> > > # The number of ticks that can pass between
> > > # sending a request and getting an acknowledgement
> > > syncLimit=2
> > > # the directory where the snapshot is stored.
> > > dataDir=/data/zookeeper/data
> > > # the port at which the clients will connect
> > > clientPort=10000
> > > clientPortAddress=172.27.31.48
> > > server.1=172.27.31.48:9900:9910
> > > server.2=172.27.31.48:9901:9911
> > > server.3=172.27.31.48:9902:9912
> > >
> >
>

limit this for security reason or network optimization.

On Wed, Jun 15, 2011 at 3:53 PM, Ted Dunning <[EMAIL PROTECTED]> wrote:

> So that other machines can contact it.
>
> Seriously, why did you have an expectation it would not bind to 0.0.0.0?
>
> The configuration file tells how to reach OTHER servers.  It doesn't
> limit which interfaces to use for incoming connections.
>
> On Wed, Jun 15, 2011 at 5:33 AM, Isaac <[EMAIL PROTECTED]> wrote:
> > why zookeeper bind 0.0.0.0 at miti-homed server?
> >
> > $ netstat -lnp | grep 99
> > ...
> > tcp        0      0 0.0.0.0:9900            0.0.0.0:*
> LISTEN
> >   16536/java
> > tcp        0      0 0.0.0.0:9910            0.0.0.0:*
> LISTEN
> >   16536/java
> > ...
> >
> > i deploy 3 zookeeper on the same server for test.
> > zoo.cfg
> > ------------------------------
> > # The number of milliseconds of each tick
> > tickTime=2000
> > # The number of ticks that the initial
> > # synchronization phase can take
> > initLimit=5
> > # The number of ticks that can pass between
> > # sending a request and getting an acknowledgement
> > syncLimit=2
> > # the directory where the snapshot is stored.
> > dataDir=/data/zookeeper/data
> > # the port at which the clients will connect
> > clientPort=10000
> > clientPortAddress=172.27.31.48
> > server.1=172.27.31.48:9900:9910
> > server.2=172.27.31.48:9901:9911
> > server.3=172.27.31.48:9902:9912
> >
>