Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper >> mail # user >> Is ZooKeeper the right tool for our needs?

Copy link to this message
Re: Is ZooKeeper the right tool for our needs?
> However, the
> ZooKeeper client connects to a ZooKeeper server at random. This is another
> indication that ZooKeeper might be the wrong tool to use.

You seed the client with the list of servers to connect to. If you
only seed it with one server, it will only connect to one server even
if there are others in the quorum, so this should not be an issue.

As for your questions, you're definitely pushing the envelope on the
standard deployment model but ZK is probably the right tool for you to
use if you care about correctness in your distributed locking. Using
ZK for distributed locking is one of its key uses. Embedding it inside
an application is not common but not unheard of. The danger with
embedding ZK is that you now have another system that can cause ZK to
perform poorly due to GC, cpu contention, etc. Is there some reason
you're concerned with running ZK as a separate VM? It won't be highly
available, but it sounds like running a standalone ZK on a separate
server than your two system servers might be the best choice for you
right now. I would probably recommend this over trying to run an
embedded quorum.


On Mon, Jun 4, 2012 at 5:44 PM, David Nickerson <

> I work for a small software company and I have been tasked with researching
> a distributed lock manager for us to use. I would appreciate any feedback
> on whether or not ZooKeeper is the right choice.
> Our software solution utilizes multiple servers, but only the Java
> application called "Administrator" needs to acquire locks. The server that
> Administrator runs on is called the System Server, which presents a web
> interface to potentially thousands of users, and is highly multi-threaded.
> So far, we have used only one System Server, and we have needed
> locks only among threads in one application. However, we are looking to
> scale the number of System Servers, and therefore need a distributed lock
> manager to be shared among them.
> We don't want to set up another server just for ZooKeeper, and we don't
> want the System Server to run another instance of the JVM just for
> ZooKeeper, so we have decided to embed the ZooKeeper server in our
> Administrator application.
> If the System Server goes down for whatever reason, we no longer have a
> need for locks, so it's not a problem if ZooKeeper goes down too. However,
> if we have two System Servers, we would like to be able to fail over to the
> second server. However, ZooKeeper requires that a majority of the ZooKeeper
> servers be up in order to run. This requirement seems strange and
> worrisome. For the time being, we can just run an additional ZooKeeper
> server on another server to give ZooKeeper the redundancy it needs.
> Another issue is that we would like to have fast locks and reduced network
> traffic by being able to read from the local ZooKeeper server. However, the
> ZooKeeper client connects to a ZooKeeper server at random. This is another
> indication that ZooKeeper might be the wrong tool to use.
> An important feature that we need is deadlock detection. I had to write
> this myself, but I was able to do so using the interface that ZooKeeper
> provides.
> We also need to be able to administrate the ZooKeeper server from our
> program. For example, if the ZooKeeper server has an error or we need to
> shut down the System Server, then the program may need to shutdown the
> ZooKeeper server or restart it. Some code to do this might be:
>    // To start the server
>    QuorumPeerMain main;
>    try {
>      QuorumPeerConfig config = new QuorumPeerConfig();
>      config.parse("ZooKeeper files\\zoo.cfg");
>      main = new QuorumPeerMain();
>      main.runFromConfig(config);
>    } catch (ConfigException e) {
>      e.printStackTrace();
>    } catch (IOException e) {
>      e.printStackTrace();
>    }
>    // To shutdown the server
>    main.quorumPeer.shutdown();
> However, "main.quorumPeer" is protected. There is very little documentation