S Ahmed 2012-05-11, 14:13
Jun Rao 2012-05-11, 14:34
-Re: since flushes are batched, is it still io intensive?
S Ahmed 2012-05-11, 15:03
What about memory? I know you guys have 24GB of ram per server?
Basically I'm juggling between going with a dedicated box (which has faster
IO), or ec2 which has slower IO but cheaper on the ram side (way cheaper!).
On Fri, May 11, 2012 at 10:34 AM, Jun Rao <[EMAIL PROTECTED]> wrote:
> It all depends on the volume of the data. At LinkedIn, we observed that the
> io load on a typical Kafka broker is not high.
> On Fri, May 11, 2012 at 7:13 AM, S Ahmed <[EMAIL PROTECTED]> wrote:
> > I was thinking (and after doing some tests on dedicated and ec2), would
> > still say kafka is io intensive?
> > Considering writes are batched every x seconds, and you have a single
> > server on a given instance, and consumers are just streaming the data in
> > sequential order (the disk head isn't jumping around), is it safe to say
> > kafka isn't that io intensive to the point that running it on ec2 should
> > just as good as dedicated hardware?
> > I was getting pretty good results on ec2 so this thought came to me...
Jun Rao 2012-05-11, 16:10
S Ahmed 2012-05-11, 16:16
Jay Kreps 2012-05-11, 23:10
S Ahmed 2012-05-12, 01:21