Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Zookeeper >> mail # user >> zookeeper on SSD


+
Jun Rao 2012-10-03, 16:59
+
Patrick Hunt 2012-10-04, 01:13
+
Jun Rao 2012-10-04, 04:12
+
Milind Parikh 2012-10-04, 04:32
+
Patrick Hunt 2012-10-04, 04:28
+
Ted Dunning 2012-10-04, 04:28
+
Andrew Purtell 2012-10-04, 04:39
Copy link to this message
-
Re: zookeeper on SSD
This is a good observation.

On Thu, Oct 4, 2012 at 5:39 AM, Andrew Purtell <[EMAIL PROTECTED]> wrote:

> Even so, I've seen in notes from attendees of Amazon's "DynamoDB For
> Developers" talks that Amazon says they found it necessary to work
> "extensively" with their SSD vendor (not stated publicly AFAIK) to
> engineer out latency spikes. I'd imagine they started with a strong
> vendor and not a low end device, but of course this is just
> speculation.
>
> On Thu, Oct 4, 2012 at 12:28 PM, Ted Dunning <[EMAIL PROTECTED]>
> wrote:
> > Yes.
> >
> > And Patrick's experience is not unexpected.  There is, however, a huge
> > variation with different types of flash memory.  The software driving the
> > flash can also result in very different experience.  The experiences that
> > he alludes to are likely with a conventional SSD packaging of flash
> driven
> > via the normal block device emulator.  That can be substantially
> > sub-optimal, depending on which vendor and configuration you use.
> >
> > On Thu, Oct 4, 2012 at 5:12 AM, Jun Rao <[EMAIL PROTECTED]> wrote:
> >
> >> Patrick,
> >>
> >> Thanks for the info. Does each ZK write wait for log being flushed to
> disk?
> >>
> >> Jun
> >>
> >> On Wed, Oct 3, 2012 at 6:13 PM, Patrick Hunt <[EMAIL PROTECTED]> wrote:
> >>
> >> > My experience with SSDs and ZK has been discouraging. SSDs have some
> >> > really terrible corner cases for latency. I've seen them take 40+
> >> > seconds (that's not a mistake - seconds) for fsync to complete. When
> >> > this happened (every few hours) all of the sessions would timeout.
> >> >
> >> > See this article:
> >> > http://storagemojo.com/2012/06/07/the-ssd-write-cliff-in-real-life/
> >> >
> >> > Patrick
> >> >
> >> > On Wed, Oct 3, 2012 at 9:59 AM, Jun Rao <[EMAIL PROTECTED]> wrote:
> >> > > Hi,
> >> > >
> >> > > Will storing the ZK commit log on SSD improve ZK write latency?
> Does a
> >> ZK
> >> > > write wait until data is flushed to disk?
> >> > >
> >> > > Thanks,
> >> > >
> >> > > Jun
> >> >
> >>
>
+
Ben Bangert 2012-10-04, 16:05
+
Flavio Junqueira 2012-10-17, 20:42
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB