Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # dev >> The consumer performance test seems to be totally broken


Copy link to this message
-
Re: The consumer performance test seems to be totally broken
Hmm, I could be wrong... Did you try this on an otherwise empty topic? Try
this:
1. Produce 100GB of messages
2. Run consumer performance test with --messages 1
3. Go get some coffee
4. Observe performance tests results with a time taken to consume 100GB of
messages but only recording a single message consumed
Here is what I mean:
scala> val l = List(1,2,3,4,5,6,7,8,9,10)
scala> val iter = l.iterator
scala> var count = 0
scala> for(i <- iter if count < 5) {
scala> iter.next
java.util.NoSuchElementException: next on empty iterator
...

This is the exact equivalent of
val l = List(1,2,3,4,5,6,7,8,9,10)
val iter = l.iterator
var count = 0
while(iter.hasNext) {
  if(iter.next < 5)
    count += 1
}

-Jay
On Sun, Feb 9, 2014 at 9:43 PM, Neha Narkhede <[EMAIL PROTECTED]>wrote:
 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB