Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Connection Timeouts


Copy link to this message
-
Re: Connection Timeouts
The request log should include produce requests too. What's the breakdown
of the consumer request that is timed out?

Thanks,

Jun
On Mon, Dec 30, 2013 at 6:40 PM, Tom Amon <[EMAIL PROTECTED]> wrote:

> That's actually easier said than done. The request logs on TRACE are
> flooded with log entries. I get about 65MB of file per day. That's on an
> idle set of brokers. There are no produce requests and only one connected
> consumer that's just sitting there. Is that normal?
>
> Thanks.
>
>
> ---------------------------------------------------------------------------
>
> These configs are reasonable and shouldn't affect consumer timeouts. Did
> you get the time breakdown from the request log?
>
>
>
> Thanks,
>
>
>
> Jun
>
>
>
>
>
> On Fri, Dec 20, 2013 at 6:14 PM, Tom Amon <[EMAIL PROTECTED]> wrote:
>
>
>
> > I figured out the cause but I'm stumped as to the reason. If the
>
> > brokers do _not_ have the replica settings in them, everything works
>
> > fine. Putting the replica settings into the config file causes all
>
> > consumers to experience a socket timeout every 30 seconds.
>
> >
>
> > Specifically here are the settings I put into the file:
>
> >
>
> > num.replica.fetchers=4
>
> > replica.fetch.max.bytes=1048576
>
> > replica.fetch.wait.max.ms=500
>
> > replica.high.watermark.checkpoint.interval.ms=5000
>
> > replica.socket.timeout.ms=30000
>
> > replica.socket.receive.buffer.bytes=65536
>
> > replica.lag.time.max.ms=10000
>
> > replica.lag.max.messages=4000
>
> >
>
> > I copied them from the kafka docs. Have I missed something important
>
> > in the documentation? These are the only properties that I changed to
>
> > cause this issue to happen. Everything else is default or specified in
>
> > the config file shipped with the kafka binaries.
>
> >
>
> > Thanks
>
> >
>
> >
>
> > ----------------------------------------------------------------------
>
> > --
>
> >
>
> > The fetch.wait.max.ms is unchanged. The request log is empty. I'll
>
> > turn it up to TRACE (it's currently on INFO) and see what it contains.
>
> >
>
> >
>
> >
>
> > Thanks
>