Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Zookeeper >> mail # user >> Recovery log on /dev/shm?


+
Ashwin Jayaprakash 2011-09-01, 20:31
+
Patrick Hunt 2011-09-01, 20:51
Copy link to this message
-
Re: Recovery log on /dev/shm?
have you benchmarked the difference between using the ram disk and
turning off sync with forceSync=no? i would think that the performance
will be similar. you will need to use the code in trunk because
https://issues.apache.org/jira/browse/ZOOKEEPER-1087 isn't in any of
the releases yet.

thanx
ben

On Thu, Sep 1, 2011 at 1:31 PM, Ashwin Jayaprakash
<[EMAIL PROTECTED]> wrote:
> This is a follow up to a question I posted in July 2010 - about whether the
> recovery log could be redirected to /dev/null or some such low latency drive
> [http://markmail.org/thread/3edpfmvn2fhyg5wv].
>
> I just found out that Linux distros with 2.6 kernel have a built-in shared
> memory file system that runs purely in memory - /dev/shm. Since this
> provides recovery across JVM restarts (but not OS restarts) wouldn't this be
> sufficient to run the ZK nodes off this drive? Replicas are on other
> machines anyway so FT/HA is already there. It's just that disk latency can
> be eliminated completely.
>
> [Apologies if you guys are already aware of /dev/shm]
>
> I ran some simple Java IO tests (not full ZK) on this drive and I wanted to
> share my results. Might be of some use to apps that require low latency:
> http://javaforu.blogspot.com/2011/09/ram-disk-is-already-in-linux-and-nobody.html
>
> Ashwin.
>
+
Mahadev Konar 2011-09-01, 22:32
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB