Raghu Angadi 2013-01-04, 15:53
Jun Rao 2013-01-04, 16:40
Raghu Angadi 2013-01-04, 17:17
-Re: How can a producer avoid slow brokers?
Neha Narkhede 2013-01-04, 18:50
queue.enqueueTimeout.ms = -1 will block the producer instead of dropping
messages. This might be useful if you have the producer being fed by scribe
On Fri, Jan 4, 2013 at 9:04 AM, Raghu Angadi <[EMAIL PROTECTED]> wrote:
> On Fri, Jan 4, 2013 at 8:39 AM, Jun Rao <[EMAIL PROTECTED]> wrote:
> > Do you know why some of the brokers are much slower than others?
> We are currently running these in a shared environment, to make things
> worse these machines have single spindle. We have to put up with that until
> we move the brokers to dedicated hardware with multiple spindles. The
> problem is a bit exaggerated in current setup.
> Even with dedicated hardware, I am expecting some variation. One slightly
> degraded disk out of 12 could reduce effective b/w on all the spindles.
> Unfortunately there will be occasional rack level network slowdowns that
> take many hours to get fixed.
> In our case, we cannot let the back pressure from slow brokers propagate
> upstream. Producers receive messages from scribe aggregators and just have
> to drop the messages it they can't write fast enough.
Raghu Angadi 2013-01-04, 19:09
Jay Kreps 2013-01-04, 23:03