-Re: slow producing on ec2
Felix GV 2012-01-10, 22:51
One order of magnitude of difference sounds about right, though, no?
I don't know. I'm by no means a Kafka expert and we don't know much about
the other variables of your situation, so it's hard to tell (at least for
me) if there could be another explanation...
On Tue, Jan 10, 2012 at 5:30 PM, Pierre-Yves Ritschard <[EMAIL PROTECTED]>wrote:
> I just didn't expect such dramatically low numbers
> On Tue, Jan 10, 2012 at 11:27 PM, Felix GV <[EMAIL PROTECTED]> wrote:
> > Maybe I'm overlooking something, but the first thing that came to my mind
> > is: wouldn't you get no network latency at all on your local box if
> > everything runs on the same machine?
> > On EC2, the network latency would bring your overal throughput down,
> > especially with a sync producer, wouldn't it?
> > --
> > Felix
> > On Tue, Jan 10, 2012 at 5:19 PM, Pierre-Yves Ritschard <[EMAIL PROTECTED]
> >> Hi,
> >> One of my cluster runs on ec2 and I noticed that I have rather slow
> >> throughput, less than 500 per second. I isolated the trouble on the
> >> following setup
> >> 1 producer instance (m1.large)
> >> 5 consumer instances also running a broker (m2.xlarge)
> >> 1 zookeeper instance (m1.large)
> >> I am using a sync producer and have 60 partitions.
> >> On my local box, with everything running I would several thousand per
> >> second.