Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Zookeeper, mail # user - Best practices for using zookeeper in an Amazon EC2 environment

Bruce Pennypacker 2013-10-16, 19:27
Copy link to this message
Re: Best practices for using zookeeper in an Amazon EC2 environment
Diego Oliveira 2013-10-16, 20:12
Hello Bruce.

     I had the same IP issue last year but I can't remember exactly what I
did but I think we used Elastic IPs. Another issue is the disk latency if
you are using EBS and have a large datasheet to be load in case of restarts
or network partition.
On Wed, Oct 16, 2013 at 4:27 PM, Bruce Pennypacker <

> We've started using an ensemble of 3 zookeepers in an EC2 environment to
> support a solr cloud setup.  I was wondering if anybody else has done so
> and if they have any recommendations, especially pertaining to instance
> sizes, DNS, etc.
> One particular question I have with regards to EC2: We had an issue a while
> back where we had to reboot one of our zookeeper instances.  Given the way
> AWS works it meant the server was assigned a new IP address.  Our current
> configuration (generated automatically by an in-house tool) uses the IP
> addresses of each server in the zookeeper configuration file.  That led to
> issues since the only way we could get the ensemble to recognize the
> rebooted server was to restart all the zookeepers so that they got the new
> IP of the rebooted instance.
> Given the nature of AWS and the likelihood that we'll need to reboot
> another zookeeper server eventually, what is the best approach to defining
> the hosts?  Can local /etc/hosts files where the we can update the IP be
> enough, or would we need to switch to using DNS for the zookeepers?  I'm
> guessing we might have to go the DNS route but I'm not 100% convinced.
> Thanks,
> -Bruce

Diego de Oliveira
System Architect
Never argue with a fool -- people might not be able to tell the difference
Jordan Zimmerman 2013-10-16, 20:13
Bruce Pennypacker 2013-10-16, 20:43