Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # dev - The consumer performance test seems to be totally broken


Copy link to this message
-
Re: The consumer performance test seems to be totally broken
Jay Kreps 2014-02-10, 17:31
Hmm, I could be wrong... Did you try this on an otherwise empty topic? Try
this:
1. Produce 100GB of messages
2. Run consumer performance test with --messages 1
3. Go get some coffee
4. Observe performance tests results with a time taken to consume 100GB of
messages but only recording a single message consumed
Here is what I mean:
scala> val l = List(1,2,3,4,5,6,7,8,9,10)
scala> val iter = l.iterator
scala> var count = 0
scala> for(i <- iter if count < 5) {
scala> iter.next
java.util.NoSuchElementException: next on empty iterator
...

This is the exact equivalent of
val l = List(1,2,3,4,5,6,7,8,9,10)
val iter = l.iterator
var count = 0
while(iter.hasNext) {
  if(iter.next < 5)
    count += 1
}

-Jay
On Sun, Feb 9, 2014 at 9:43 PM, Neha Narkhede <[EMAIL PROTECTED]>wrote: