Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Consumer stops consuming after 5 gb


+
Anurup Raveendran 2013-07-23, 14:59
+
Jun Rao 2013-07-23, 15:37
+
Samir Madhavan 2013-07-23, 16:56
+
Samir Madhavan 2013-07-23, 17:18
Copy link to this message
-
Re: Consumer stops consuming after 5 gb
Samir,

When you observe that the consumer hangs, could you send around the thread
dump. Also can you check the value of the FetchRate mbean ?

Thanks,
Neha
On Tue, Jul 23, 2013 at 10:17 AM, Samir Madhavan <
[EMAIL PROTECTED]> wrote:

> We just observed that the consumer is occupying almost complete RAM which
> is 4Gb.
>
>
> On Tue, Jul 23, 2013 at 10:25 PM, Samir Madhavan <
> [EMAIL PROTECTED]> wrote:
>
> > One of the common pattern we observed when we ran consumer from start
> > multiple times is that it catches up till the data that producer has
> > produced and then it just gets hung. If we kill it and then start it
> again,
> > it starts consuming and then again it just gets hung. We have added the
> try
> > catch statements but there seems to be no error coming.
> >
> >
> > On Tue, Jul 23, 2013 at 9:06 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
> >
> >> Not sure about the error. However, your consumer seems to be lagging.
> Are
> >> the consumer offsets moving at all? If not, are your consumer threads
> >> still
> >> alive? You probably want to try/catch the consumer code to see if there
> is
> >> any unexpected exceptions.
> >>
> >> Thanks,
> >>
> >> Jun
> >>
> >>
> >> On Tue, Jul 23, 2013 at 7:58 AM, Anurup Raveendran <
> >> [EMAIL PROTECTED]> wrote:
> >>
> >> > After consuming about 5 gb of messages , it stopped consuming and it
> got
> >> > stuck at a particular offset. After running the following command I
> got
> >> an
> >> > error regarding the broker.
> >> >
> >> >
> >> >
> >> >
> >> > $ bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --group
> >> > air-DummyProductionConsumerGroup-After --zkconnect 172.16.18.65:2181
> >> --topic
> >> > air
> >> >
> >> >
> >> >
> >> > [2013-07-23 20:19:42,748] INFO Starting ZkClient event thread.
> >> > (org.I0Itec.zkclient.ZkEventThread)
> >> > [2013-07-23 20:19:42,757] INFO Client
> >> > environment:zookeeper.version=3.3.3-1203054, built on 11/17/2011 05:47
> >> GMT
> >> > (org.apache.zookeeper.ZooKeeper)
> >> > [2013-07-23 20:19:42,757] INFO Client environment:host.name=localhost
> >> > (org.apache.zookeeper.ZooKeeper)
> >> > [2013-07-23 20:19:42,757] INFO Client
> environment:java.version=1.7.0_21
> >> > (org.apache.zookeeper.ZooKeeper)
> >> > [2013-07-23 20:19:42,757] INFO Client environment:java.vendor=Oracle
> >> > Corporation (org.apache.zookeeper.ZooKeeper)
> >> > [2013-07-23 20:19:42,757] INFO Client
> >> > environment:java.home=/usr/java/jdk1.7.0_21/jre
> >> > (org.apache.zookeeper.ZooKeeper)
> >> > [2013-07-23 20:19:42,757] INFO Client
> >> >
> >> >
> >>
> environment:java.class.path=:bin/../project/boot/scala-2.8.0/lib/scala-compiler.jar:bin/../project/boot/scala-2.8.0/lib/scala-library.jar:bin/../core/target/scala_2.8.0/kafka-0.7.2.jar:bin/../core/lib/*.jar:bin/../perf/target/scala_2.8.0/kafka-perf-0.7.2.jar:bin/../core/lib_managed/scala_2.8.0/compile/jopt-simple-3.2.jar:bin/../core/lib_managed/scala_2.8.0/compile/log4j-1.2.15.jar:bin/../core/lib_managed/scala_2.8.0/compile/snappy-java-1.0.4.1.jar:bin/../core/lib_managed/scala_2.8.0/compile/zkclient-0.1.jar:bin/../core/lib_managed/scala_2.8.0/compile/zookeeper-3.3.4.jar
> >> > (org.apache.zookeeper.ZooKeeper)
> >> > [2013-07-23 20:19:42,758] INFO Client
> >> >
> >> >
> >>
> environment:java.library.path=/usr/local/oracle:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> >> > (org.apache.zookeeper.ZooKeeper)
> >> > [2013-07-23 20:19:42,758] INFO Client environment:java.io.tmpdir=/tmp
> >> > (org.apache.zookeeper.ZooKeeper)
> >> > [2013-07-23 20:19:42,758] INFO Client environment:java.compiler=<NA>
> >> > (org.apache.zookeeper.ZooKeeper)
> >> > [2013-07-23 20:19:42,758] INFO Client environment:os.name=Linux
> >> > (org.apache.zookeeper.ZooKeeper)
> >> > [2013-07-23 20:19:42,758] INFO Client environment:os.arch=amd64
> >> > (org.apache.zookeeper.ZooKeeper)
> >> > [2013-07-23 20:19:42,758] INFO Client
> >> > environment:os.version=2.6.32-279.22.1.el6.x86_64

 
+
Samir Madhavan 2013-07-29, 10:06