Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Zookeeper, mail # user - Recovery log on /dev/shm?

Ashwin Jayaprakash 2011-09-01, 20:31
Patrick Hunt 2011-09-01, 20:51
Copy link to this message
Re: Recovery log on /dev/shm?
Benjamin Reed 2011-09-01, 21:24
have you benchmarked the difference between using the ram disk and
turning off sync with forceSync=no? i would think that the performance
will be similar. you will need to use the code in trunk because
https://issues.apache.org/jira/browse/ZOOKEEPER-1087 isn't in any of
the releases yet.


On Thu, Sep 1, 2011 at 1:31 PM, Ashwin Jayaprakash
> This is a follow up to a question I posted in July 2010 - about whether the
> recovery log could be redirected to /dev/null or some such low latency drive
> [http://markmail.org/thread/3edpfmvn2fhyg5wv].
> I just found out that Linux distros with 2.6 kernel have a built-in shared
> memory file system that runs purely in memory - /dev/shm. Since this
> provides recovery across JVM restarts (but not OS restarts) wouldn't this be
> sufficient to run the ZK nodes off this drive? Replicas are on other
> machines anyway so FT/HA is already there. It's just that disk latency can
> be eliminated completely.
> [Apologies if you guys are already aware of /dev/shm]
> I ran some simple Java IO tests (not full ZK) on this drive and I wanted to
> share my results. Might be of some use to apps that require low latency:
> http://javaforu.blogspot.com/2011/09/ram-disk-is-already-in-linux-and-nobody.html
> Ashwin.
Mahadev Konar 2011-09-01, 22:32