-Re: High level consumer Blocked when there is still message in topic
Jun Rao 2013-11-13, 05:28
What's the max lag (reported in JMX) in the consumer? Can the consumer keep
up with the incoming data rate?
On Tue, Nov 12, 2013 at 7:19 PM, 李帅 <[EMAIL PROTECTED]> wrote:
> I use Kafka 0.8 high level consumer reads message from topic
> stream, 3 replica and 10 paritions.
> When I use 10 threads read the stream and runing for some time (one
> hour or one day),
> some threads block at m_stream.iterator().hasNext(), but the parition
> still has lots of messages.
> I check consumer's fetch.message.max.bytes and broker's
> message.max.bytes, there is no
> message size bigger than these values.
> The consumer configure is
> props.put("zookeeper.session.timeout.ms", "4000");
> props.put("zookeeper.sync.time.ms", "200");
> props.put("auto.commit.interval.ms", "1000");
> Please give me some option about how to avoid consumer block.
> Is there some configure parameter can fix this problem.