Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper >> mail # user >> Newbie design question


Copy link to this message
-
Re: Newbie design question
Run an ensemble of ZK machines. Each job node just needs to use a zookeeper
client library to pull jobs down.
You do not want to run a large ensemble of ZK nodes unless absolutely
necessary. The more machines in a ZK ensemble, the slower writes are able
to be processed. The only reason for going larger than 3 is really to
enable many more concurrent client connections (on the order of tens of
thousands). Sounds like you are not in that situation, so a 3 node ensemble
is probably best for you.

C

On Mon, Mar 5, 2012 at 6:58 AM, Joe Gamache <[EMAIL PROTECTED]> wrote:

> Hello folks,
>
> I skimmed the archives, but couldn't find a way to search them, does that
> exist?
>
> So I am new to zookeeper and was hoping to get some early guidance.  The
> first little part of what I need to implement is fairly straight-forward.
>  I have a list of requests.  Each request needs to be issued out to a
> different machine and monitored.   Ok, it's very straight-forward!  :)
>  There are no dependencies between these jobs.  There is to another set of
> jobs, but I don't have to deal with that for this first step.  There will
> probably be on the order of fifty machines.   Looking over the
> documentation and examples, it seems like I can just use a Queue, Executor,
> and Monitor....
>
> My question is do I need/want zookeeper on all of the nodes or just 3,5,
> or 7 (as an ensemble)?   Is there a known tradeoff between the two
> approaches - all nodes or small subset?
>
> Sorry if this or similar questions have already been asked, but thanks for
> any pointers.
>
> Joe
>
>