Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - my performance numbers


Copy link to this message
-
Re: my performance numbers
S Ahmed 2012-05-07, 13:33
Actually I was thinking, since my payloads sizes are larger, it would make
sense to flush to disk more often right?  Since I only have 4GB ram on the
server.

What effect does the poll time to check if a flush is required have?    Is
it an expensive operation, i.e. does it walk the entire collection and
determine the size?

On Sat, May 5, 2012 at 3:22 PM, S Ahmed <[EMAIL PROTECTED]> wrote:

> These are the performance #'s I got on a dedicated server, everthing
> (zookeeper, kafka server) is running on the same server:
>
> bin/kafka-producer-perf-test.sh --brokerinfo broker.list=0:localhost:9092
> --messages 1000000 --topic test --threads 10 --message-size=30720
> [2012-05-05 14:03:31,344] INFO Creating sync producer for broker id = 0 at
> localhost:9092 (kafka.producer.ProducerPool)
> [2012-05-05 14:03:31,345] INFO Creating sync producer for broker id = 0 at
> localhost:9092 (kafka.producer.ProducerPool)
> [2012-05-05 14:03:31,346] INFO Creating sync producer for broker id = 0 at
> localhost:9092 (kafka.producer.ProducerPool)
> [2012-05-05 14:03:31,346] INFO Creating sync producer for broker id = 0 at
> localhost:9092 (kafka.producer.ProducerPool)
> [2012-05-05 14:03:31,346] INFO Creating sync producer for broker id = 0 at
> localhost:9092 (kafka.producer.ProducerPool)
> [2012-05-05 14:03:31,347] INFO Creating sync producer for broker id = 0 at
> localhost:9092 (kafka.producer.ProducerPool)
> [2012-05-05 14:03:31,347] INFO Creating sync producer for broker id = 0 at
> localhost:9092 (kafka.producer.ProducerPool)
> [2012-05-05 14:03:31,348] INFO Creating sync producer for broker id = 0 at
> localhost:9092 (kafka.producer.ProducerPool)
> [2012-05-05 14:03:31,352] INFO Creating sync producer for broker id = 0 at
> localhost:9092 (kafka.producer.ProducerPool)
> [2012-05-05 14:03:31,353] INFO Messages per thread = 500
> (kafka.tools.ProducerPerformance$SyncProducerThread)
> [2012-05-05 14:03:31,354] INFO Messages per thread = 500
> (kafka.tools.ProducerPerformance$SyncProducerThread)
> [2012-05-05 14:03:31,354] INFO Messages per thread = 500
> (kafka.tools.ProducerPerformance$SyncProducerThread)
> [2012-05-05 14:03:31,354] INFO Messages per thread = 500
> (kafka.tools.ProducerPerformance$SyncProducerThread)
> [2012-05-05 14:03:31,354] INFO Messages per thread = 500
> (kafka.tools.ProducerPerformance$SyncProducerThread)
> [2012-05-05 14:03:31,354] INFO Messages per thread = 500
> (kafka.tools.ProducerPerformance$SyncProducerThread)
> [2012-05-05 14:03:31,354] INFO Messages per thread = 500
> (kafka.tools.ProducerPerformance$SyncProducerThread)
> [2012-05-05 14:03:31,354] INFO Messages per thread = 500
> (kafka.tools.ProducerPerformance$SyncProducerThread)
> [2012-05-05 14:03:31,354] INFO Creating sync producer for broker id = 0 at
> localhost:9092 (kafka.producer.ProducerPool)
> [2012-05-05 14:03:31,357] INFO Messages per thread = 500
> (kafka.tools.ProducerPerformance$SyncProducerThread)
> [2012-05-05 14:03:31,358] INFO Messages per thread = 500
> (kafka.tools.ProducerPerformance$SyncProducerThread)
> [2012-05-05 14:03:31,632] INFO Connected to localhost:9092 for producing
> (kafka.producer.SyncProducer)
> [2012-05-05 14:03:31,632] INFO Connected to localhost:9092 for producing
> (kafka.producer.SyncProducer)
> [2012-05-05 14:03:31,632] INFO Connected to localhost:9092 for producing
> (kafka.producer.SyncProducer)
> [2012-05-05 14:03:31,632] INFO Connected to localhost:9092 for producing
> (kafka.producer.SyncProducer)
> [2012-05-05 14:03:31,632] INFO Connected to localhost:9092 for producing
> (kafka.producer.SyncProducer)
> [2012-05-05 14:03:31,632] INFO Connected to localhost:9092 for producing
> (kafka.producer.SyncProducer)
> [2012-05-05 14:03:31,632] INFO Connected to localhost:9092 for producing
> (kafka.producer.SyncProducer)
> [2012-05-05 14:03:31,632] INFO Connected to localhost:9092 for producing
> (kafka.producer.SyncProducer)
> [2012-05-05 14:03:31,632] INFO Connected to localhost:9092 for producing
> (kafka.producer.SyncProducer)