Kafka, mail # user - Re: Kafka consumer not consuming events - 2013-07-09, 15:38
Solr & Elasticsearch trainings in New York & San Fransisco [more info][hide]
 Search Hadoop and all its subprojects:

Switch to Threaded View
Copy link to this message
-
Re: Kafka consumer not consuming events
Hello Jun,

Please see my comments inline.

On 09-Jul-2013, at 8:32 PM, Jun Rao <[EMAIL PROTECTED]> wrote:

No, we kept dedicated consumer listening to the topic in question.
We did this because this queue processes huge amounts of data.

Yes. We can see all the threads in the thread dump.
We have ensured that the threads do not die due to an Exception.

Please look at the stack trace below. We see all the threads waiting like this:

"event_queue@150" prio=10 tid=0x00007eff28e41800 nid=0x31f9 waiting on condition [0x00007efedae6d000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x0000000640248618> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
        at kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:60)
        at kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:32)
        at kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:59)
        at kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:51)
        at com.spr.messageprocessor.KafkaStreamRunnable.run(KafkaStreamRunnable.java:49)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:662)

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB