Search Hadoop and all its sub project:

Switch to Threaded View
Subject: Why does the high level consumer block, or rather where does it?
I'm trying to trace through the codebase and figure out where exactly the
block occurs in the high level consumer?

public void run() {
     ConsumerIterator<byte[], byte[]> it = m_stream.iterator();
     while (it.hasNext())
         System.out.println("Thread " + m_threadNumber + ": " + new
     System.out.println("Shutting down Thread: " + m_threadNumber);

So from what I understand, the will block if there are
no new messages for this particular topic/partion correct?

Just to understand, can someone clarify where in the kafka source this
block occurs, i.e. the broker that this consumer is connected to will keep
a socket connection open to this consumer and block until a new message
that is owned by this consumer thread arrives and then pushes it to the
consumer to process.

Is it at the iterator level somewhere?

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB