Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Zookeeper >> mail # user >> curator leader election and thread usage


+
john doe 2012-05-07, 12:13
+
Patrick Hunt 2012-05-09, 21:51
Copy link to this message
-
Re: curator leader election and thread usage
I've decided to add an alternate version of leader selection that doesn't
use threads. It will behave somewhat like a CountDownLatch so I'm calling
it LeaderLatch. I'll report back (only on the Curator list) in the next
day or so when it's available.

-Jordan

On 5/9/12 2:51 PM, "Patrick Hunt" <[EMAIL PROTECTED]> wrote:

>This might be better directed to the Curator list (ccing them).
>
>I'm guessing this "thread per leader" is related to a curator pattern?
>Using the ZK api directly you should be able to use a single thread to
>hold multiple leaderships. Perhaps there's a way to get this behavior
>using Curator.
>
>Patrick
>
>On Mon, May 7, 2012 at 5:13 AM, john doe <[EMAIL PROTECTED]> wrote:
>> Hello,
>>
>> I am currently using curator to manage leader election (which I find
>>really
>> nice btw), and I have a question:
>>
>> I am using leaders to manage shards of a database. When a node (=a jvm)
>>is
>> started, it is told what shards it could manage if it is elected for
>>this
>> shard. A node can be elected for multiple shards.
>>
>> The issue I got is that a node typically manages hundreds of shards. Not
>> sure I understood correctly, but current leader election receipe (as
>> implemented in curator) results in the creation of hundreds of threads,
>> which I keep sleeping because actual processing is done by a separate
>> thread pool.
>>
>> Is there a way not to use that many threads, given that when a node is
>> elected for a shard it will never release its leadership?
>>
>> Thanks
>>
>> John
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB