With all the community help I was able to integrate ZK to Apache Airavata to achieve fault-tolerance and it was a very interesting experience to work with ZK. It works as it explains without any issue.
Now I have an issue how to bundle and ship it. Currently what I have asked the community is to start ZK instance then run our services. Personally I like that approach and its much cleaner and in production we can cluster both Airavata and ZK. But for the curiosity I want to know is there a better way to bundle like an embedded zk which is stable enough for day today development or there is a better way to do it.
 Regards Lahiru
System Analyst Programmer PTI Lab Indiana University
I had embedded ZooKeeper server without any issues but not in the production cluster. I feel you can use it for day to day development phase. Its being used in ZK unit tests, please refer org.apache.zookeeper.test.QuorumUtil or org.apache.zookeeper.server.quorum.QuorumPeerTestBase.
I've noticed few cases, this may be useful to you.
1) ZK server code has System.exit() which may affect your service. 2) Better to redirect ZK logs to a separate log file which will help in debugging issues independently. 3) Observe the network traffic and GC, this may affect ZK server communications and resulting in failures. 4) In general, it would be difficult to restart your service or ZK server without affecting each other. -Rakesh