Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Zookeeper >> mail # user >> Newbie design question


+
Joe Gamache 2012-03-05, 11:58
Copy link to this message
-
Re: Newbie design question
Run an ensemble of ZK machines. Each job node just needs to use a zookeeper
client library to pull jobs down.
You do not want to run a large ensemble of ZK nodes unless absolutely
necessary. The more machines in a ZK ensemble, the slower writes are able
to be processed. The only reason for going larger than 3 is really to
enable many more concurrent client connections (on the order of tens of
thousands). Sounds like you are not in that situation, so a 3 node ensemble
is probably best for you.

C

On Mon, Mar 5, 2012 at 6:58 AM, Joe Gamache <[EMAIL PROTECTED]> wrote:

> Hello folks,
>
> I skimmed the archives, but couldn't find a way to search them, does that
> exist?
>
> So I am new to zookeeper and was hoping to get some early guidance.  The
> first little part of what I need to implement is fairly straight-forward.
>  I have a list of requests.  Each request needs to be issued out to a
> different machine and monitored.   Ok, it's very straight-forward!  :)
>  There are no dependencies between these jobs.  There is to another set of
> jobs, but I don't have to deal with that for this first step.  There will
> probably be on the order of fifty machines.   Looking over the
> documentation and examples, it seems like I can just use a Queue, Executor,
> and Monitor....
>
> My question is do I need/want zookeeper on all of the nodes or just 3,5,
> or 7 (as an ensemble)?   Is there a known tradeoff between the two
> approaches - all nodes or small subset?
>
> Sorry if this or similar questions have already been asked, but thanks for
> any pointers.
>
> Joe
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB