Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Kafka 0.7 performance compared to bare metal


Copy link to this message
-
Re: Kafka 0.7 performance compared to bare metal
You are maxing out the single consumer thread.
On Aug 30, 2013 1:35 AM, "Rafael Bagmanov" <[EMAIL PROTECTED]> wrote:

> Hi,
>
> I am trying to understand how fast is kafka 0.7 compared to what I can get
> from hard drive. In essence I have 3 questions.
>
> In all tests below, I'm using single broker with single one-partitioned
> topic. Kafka perf tests have been run in 2 deployment configs:
> - broker, perf-test on same host
> - broker, perf-test on different hosts (the results are practically the
> same, so wont post them here)
>
>
> I'm using FIO(http://freecode.com/projects/fio) to benchmark speed of hard
> drives.
>
> Hardware I'm using:
> 1) m1.xlarge with ephemeral storage, 4 core cpu, 16 GB ram
> 2) hi1.4xlarge  with SSD, 16 core cpu, 64 GB ram
> 3) desktop machine with 7200 rpm sata, 4 core cpu, 8 GB ram
>
> Kafka broker config:
> Oracle jdk 1.6.0_38,  -Xmx2048
>
> socket.send.buffer=16777216
> socket.receive.buffer=16777216
> max.socket.request.bytes=104857600
> log.flush.interval=10000
> log.default.flush.interval.ms=1000
> log.default.flush.scheduler.interval.ms=1000
> num.threads=[num of cores]
>
>
> For kafka-producer-perf-test I'm assuming that IO access pattern is
> sequential write.
>
> Here is the test I ran with FIO:
>
> [sequential-write]
> rw=write
> size=50G
> ioengine=sync
> numjobs=1
> directory=/tmp/fio
> filename=redo01.log
>
>
> Here is kafka performance test:
>
> ./bin/kafka-producer-perf-test.sh -topic "perf" --batch-size 3000
> --messages 50000000 --message-size 1300 --brokerinfo
> broker.list=0:host:9092 --threads [number-of-cores]
>
>
> ----------------------------------------------------------------------------------------
> |           |   m1.xlarge            |    hi1.4xlarge       |  desktop
>  |
>
>  ----------------------------------------------------------------------------------------
> |  kafka  |     41 MB/s           |      217 MB/s       |     42 MB/s   |
>
>  -----------------------------------------------------------------------------------------
> |  fio      |     106 MB/s          |      377 MB/s       |    74 MB/s   |
>
> ----------------------------------------------------------------------------------------
>
>
> Question 1: The proportion (~1/2) is pretty stable against different kind
> of hardware I've tried. Is it as expected? Can something be done to improve
> this?
>
> I've tried to play with:
> log.flush.interval=10000
> log.default.flush.interval.ms=1000
> log.default.flush.scheduler.interval.ms=1000
>
> Like increasing 10 times, or decreasing 10 times, but haven't seen much of
> a difference in IO  throughput
>
> The other thing that bugs me much more is that kafka consumer speed on cold
> IO cache is like 5-50 times slower from what I can get with "sequential
> read" fio test.
>
> For kafka-consumer-perf-test I'm assuming that IO access pattern is
> sequential read.
>
> Here is FIO test:
>
> [sequential-read]
> rw=read
> size=50G
> ioengine=sync   # I know that kafka use sendfile, but sync should be
> slower, right?
> numjobs=1
>  directory=/tmp/fio
> filename=redo01.log
>
> Here what I'm doing with kafka-consumer-perf-test:
>
> kafka-consumer-perf-test.sh -topic "perf" --messages 50000000 --zookeeper
> host:2181 --threads 1 --socket-buffer-size 16777216 --fetch-size 16777216
>
> The broker config is  the same.
>
> I'm dropping IO cache before running tests: echo 3 >
> /proc/sys/vm/drop_caches
>
>
> -----------------------------------------------------------------------------------------------
> |           |   m1.xlarge            |    hi1.4xlarge              |
>  desktop    |
>
>  ---------------------------------------------------------------------------------------------
> |  kafka  |    25   MB/s           |     10  MB/s   (???)    |   20  MB/s
>  |
>
>  ---------------------------------------------------------------------------------------------
> |  fio      |   130   MB/s          |     450  MB/s             |    67
> MB/s  |
>
> ---------------------------
 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB