Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper >> mail # user >> zookeeper on SSD


Copy link to this message
-
Re: zookeeper on SSD
This is a good observation.

On Thu, Oct 4, 2012 at 5:39 AM, Andrew Purtell <[EMAIL PROTECTED]> wrote:

> Even so, I've seen in notes from attendees of Amazon's "DynamoDB For
> Developers" talks that Amazon says they found it necessary to work
> "extensively" with their SSD vendor (not stated publicly AFAIK) to
> engineer out latency spikes. I'd imagine they started with a strong
> vendor and not a low end device, but of course this is just
> speculation.
>
> On Thu, Oct 4, 2012 at 12:28 PM, Ted Dunning <[EMAIL PROTECTED]>
> wrote:
> > Yes.
> >
> > And Patrick's experience is not unexpected.  There is, however, a huge
> > variation with different types of flash memory.  The software driving the
> > flash can also result in very different experience.  The experiences that
> > he alludes to are likely with a conventional SSD packaging of flash
> driven
> > via the normal block device emulator.  That can be substantially
> > sub-optimal, depending on which vendor and configuration you use.
> >
> > On Thu, Oct 4, 2012 at 5:12 AM, Jun Rao <[EMAIL PROTECTED]> wrote:
> >
> >> Patrick,
> >>
> >> Thanks for the info. Does each ZK write wait for log being flushed to
> disk?
> >>
> >> Jun
> >>
> >> On Wed, Oct 3, 2012 at 6:13 PM, Patrick Hunt <[EMAIL PROTECTED]> wrote:
> >>
> >> > My experience with SSDs and ZK has been discouraging. SSDs have some
> >> > really terrible corner cases for latency. I've seen them take 40+
> >> > seconds (that's not a mistake - seconds) for fsync to complete. When
> >> > this happened (every few hours) all of the sessions would timeout.
> >> >
> >> > See this article:
> >> > http://storagemojo.com/2012/06/07/the-ssd-write-cliff-in-real-life/
> >> >
> >> > Patrick
> >> >
> >> > On Wed, Oct 3, 2012 at 9:59 AM, Jun Rao <[EMAIL PROTECTED]> wrote:
> >> > > Hi,
> >> > >
> >> > > Will storing the ZK commit log on SSD improve ZK write latency?
> Does a
> >> ZK
> >> > > write wait until data is flushed to disk?
> >> > >
> >> > > Thanks,
> >> > >
> >> > > Jun
> >> >
> >>
>