Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper >> mail # user >> forceSync=no


Copy link to this message
-
Re: forceSync=no
Raj,
 Are you seeing any perf improvements by using that setting? I had run
some benchmarks and didnt find much improvement even if I did
forceSync = no.

thanks
mahadev

On Fri, Jun 15, 2012 at 12:45 PM, Raj N <[EMAIL PROTECTED]> wrote:
> Thanks Patrick. So back to my original question. Will forceSync=no cause a
> zookeeper node to not start up cleanly if it crashed? It doesn't happen all
> the time. But it happened to me once. I had to cleanup all the transaction
> logs on all the nodes and then bring the nodes back up. For my use case I
> can tolerate losing data in the worst case scenario since I can regenerate
> all the data. But I should be able to tolerate a single point of failure as
> long as the node comes back up cleanly. Can zookeeper recover from a
> corrupt transaction log using existing snapshots and then replaying
> messages from its peers?
>
> Thanks
> -Raj
>
> On Fri, Jun 15, 2012 at 2:17 PM, Patrick Hunt <[EMAIL PROTECTED]> wrote:
>
>> For a single node failure I believe so.
>>
>> Patrick
>>
>> On Thu, Jun 14, 2012 at 11:56 AM, Raj N <[EMAIL PROTECTED]> wrote:
>> > Sorry, I should have been more specific. By corrupt, I mean that the
>> > zookeeper node doesn't come back up on a restart. I would have imagined
>> > that zookeeper would sync the lost transactions from its peers. I agree I
>> > will have a problem if I have multiple failures. But for a single node
>> > failure in a 3-node ensemble, I should be able to recover even if
>> > forceSync=no.
>> >
>> > Thanks
>> > -Raj
>> >
>> > On Thu, Jun 14, 2012 at 1:03 PM, Jonathan Simms <[EMAIL PROTECTED]>
>> wrote:
>> >
>> >> There's a big warning in the documentation that says that's a
>> possibility.
>> >> If you don't force both Java and the OS to flush their IO buffers to
>> disk,
>> >> then you have no guarantees that your data is consistent.
>> >>
>> >> On 6/14/12 10:56 AM, "Raj N" <[EMAIL PROTECTED]> wrote:
>> >>
>> >> >Are you guys aware of any issues with forceSync=no that could cause the
>> >> >transaction log to get corrupted on a zookeeper crash.
>> >> >
>> >> >Thanks
>> >> >-Raj
>> >>
>> >>
>> >>
>>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB