Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> my performance numbers


Copy link to this message
-
Re: my performance numbers
Actually I was thinking, since my payloads sizes are larger, it would make
sense to flush to disk more often right?  Since I only have 4GB ram on the
server.

What effect does the poll time to check if a flush is required have?    Is
it an expensive operation, i.e. does it walk the entire collection and
determine the size?

On Sat, May 5, 2012 at 3:22 PM, S Ahmed <[EMAIL PROTECTED]> wrote:

> These are the performance #'s I got on a dedicated server, everthing
> (zookeeper, kafka server) is running on the same server:
>
> bin/kafka-producer-perf-test.sh --brokerinfo broker.list=0:localhost:9092
> --messages 1000000 --topic test --threads 10 --message-size=30720
> [2012-05-05 14:03:31,344] INFO Creating sync producer for broker id = 0 at
> localhost:9092 (kafka.producer.ProducerPool)
> [2012-05-05 14:03:31,345] INFO Creating sync producer for broker id = 0 at
> localhost:9092 (kafka.producer.ProducerPool)
> [2012-05-05 14:03:31,346] INFO Creating sync producer for broker id = 0 at
> localhost:9092 (kafka.producer.ProducerPool)
> [2012-05-05 14:03:31,346] INFO Creating sync producer for broker id = 0 at
> localhost:9092 (kafka.producer.ProducerPool)
> [2012-05-05 14:03:31,346] INFO Creating sync producer for broker id = 0 at
> localhost:9092 (kafka.producer.ProducerPool)
> [2012-05-05 14:03:31,347] INFO Creating sync producer for broker id = 0 at
> localhost:9092 (kafka.producer.ProducerPool)
> [2012-05-05 14:03:31,347] INFO Creating sync producer for broker id = 0 at
> localhost:9092 (kafka.producer.ProducerPool)
> [2012-05-05 14:03:31,348] INFO Creating sync producer for broker id = 0 at
> localhost:9092 (kafka.producer.ProducerPool)
> [2012-05-05 14:03:31,352] INFO Creating sync producer for broker id = 0 at
> localhost:9092 (kafka.producer.ProducerPool)
> [2012-05-05 14:03:31,353] INFO Messages per thread = 500
> (kafka.tools.ProducerPerformance$SyncProducerThread)
> [2012-05-05 14:03:31,354] INFO Messages per thread = 500
> (kafka.tools.ProducerPerformance$SyncProducerThread)
> [2012-05-05 14:03:31,354] INFO Messages per thread = 500
> (kafka.tools.ProducerPerformance$SyncProducerThread)
> [2012-05-05 14:03:31,354] INFO Messages per thread = 500
> (kafka.tools.ProducerPerformance$SyncProducerThread)
> [2012-05-05 14:03:31,354] INFO Messages per thread = 500
> (kafka.tools.ProducerPerformance$SyncProducerThread)
> [2012-05-05 14:03:31,354] INFO Messages per thread = 500
> (kafka.tools.ProducerPerformance$SyncProducerThread)
> [2012-05-05 14:03:31,354] INFO Messages per thread = 500
> (kafka.tools.ProducerPerformance$SyncProducerThread)
> [2012-05-05 14:03:31,354] INFO Messages per thread = 500
> (kafka.tools.ProducerPerformance$SyncProducerThread)
> [2012-05-05 14:03:31,354] INFO Creating sync producer for broker id = 0 at
> localhost:9092 (kafka.producer.ProducerPool)
> [2012-05-05 14:03:31,357] INFO Messages per thread = 500
> (kafka.tools.ProducerPerformance$SyncProducerThread)
> [2012-05-05 14:03:31,358] INFO Messages per thread = 500
> (kafka.tools.ProducerPerformance$SyncProducerThread)
> [2012-05-05 14:03:31,632] INFO Connected to localhost:9092 for producing
> (kafka.producer.SyncProducer)
> [2012-05-05 14:03:31,632] INFO Connected to localhost:9092 for producing
> (kafka.producer.SyncProducer)
> [2012-05-05 14:03:31,632] INFO Connected to localhost:9092 for producing
> (kafka.producer.SyncProducer)
> [2012-05-05 14:03:31,632] INFO Connected to localhost:9092 for producing
> (kafka.producer.SyncProducer)
> [2012-05-05 14:03:31,632] INFO Connected to localhost:9092 for producing
> (kafka.producer.SyncProducer)
> [2012-05-05 14:03:31,632] INFO Connected to localhost:9092 for producing
> (kafka.producer.SyncProducer)
> [2012-05-05 14:03:31,632] INFO Connected to localhost:9092 for producing
> (kafka.producer.SyncProducer)
> [2012-05-05 14:03:31,632] INFO Connected to localhost:9092 for producing
> (kafka.producer.SyncProducer)
> [2012-05-05 14:03:31,632] INFO Connected to localhost:9092 for producing
> (kafka.producer.SyncProducer)
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB