Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka, mail # user - seeing poor consumer performance in 0.7.2


+
Andrew Neilson 2013-04-21, 09:00
+
Neha Narkhede 2013-04-21, 15:50
+
Andrew Neilson 2013-04-21, 19:23
Copy link to this message
-
Re: seeing poor consumer performance in 0.7.2
Jun Rao 2013-04-22, 03:51
Is your network shared? Is so, another possibility is that some other apps
are consuming the bandwidth.

Thanks,

Jun
On Sun, Apr 21, 2013 at 12:23 PM, Andrew Neilson <[EMAIL PROTECTED]>wrote:

> Thanks very much for the reply Neha! So I swapped out the consumer that
> processes the messages with one that just prints them. It does indeed
> achieve a much better rate at peaks but can still nearly zero out (if not
> completely zero out). I plotted the messages printed in graphite to show
> the behaviour I'm seeing (this is messages printed per second):
>
>
> https://www.dropbox.com/s/7u7uyrefw6inetu/Screen%20Shot%202013-04-21%20at%2011.44.38%20AM.png
>
> The peaks are over ten thousand per second and the troughs can go below 10
> per second just prior to another peak. I know that there are plenty of
> messages available because the ones currently being processed are still
> from Friday afternoon, so this may or may not have something to do with
> this pattern.
>
> Is there anything I can do to avoid the periods of lower performance?
> Ideally I would be processing messages as soon as they are written.
>
>
> On Sun, Apr 21, 2013 at 8:49 AM, Neha Narkhede <[EMAIL PROTECTED]
> >wrote:
>
> > Some of the reasons a consumer is slow are -
> > 1. Small fetch size
> > 2. Expensive message processing
> >
> > Are you processing the received messages in the consumer ? Have you
> > tried running console consumer for this topic and see how it performs
> > ?
> >
> > Thanks,
> > Neha
> >
> > On Sun, Apr 21, 2013 at 1:59 AM, Andrew Neilson <[EMAIL PROTECTED]>
> > wrote:
> > > I am currently running a deployment with 3 brokers, 3 ZK, 3 producers,
> 2
> > > consumers, and 15 topics. I should first point out that this is my
> first
> > > project using Kafka ;). The issue I'm seeing is that the consumers are
> > only
> > > processing about 15 messages per second from what should be the largest
> > > topic it is consuming (we're sending 200-400 ~300 byte messages per
> > second
> > > to this topic). I should note that I'm using a high level ZK consumer
> and
> > > ZK 3.4.3.
> > >
> > > I have a strong feeling I have not configured things properly so I
> could
> > > definitely use some guidance. Here is my broker configuration:
> > >
> > > brokerid=1
> > > port=9092
> > > socket.send.buffer=1048576
> > > socket.receive.buffer=1048576
> > > max.socket.request.bytes=104857600
> > > log.dir=/home/kafka/data
> > > num.partitions=1
> > > log.flush.interval=10000
> > > log.default.flush.interval.ms=1000
> > > log.default.flush.scheduler.interval.ms=1000
> > > log.retention.hours=168
> > > log.file.size=536870912
> > > enable.zookeeper=true
> > > zk.connect=XXX
> > > zk.connectiontimeout.ms=1000000
> > >
> > > Here is my producer config:
> > >
> > > zk.connect=XXX
> > > producer.type=async
> > > compression.codec=0
> > >
> > > Here is my consumer config:
> > >
> > > zk.connect=XXX
> > > zk.connectiontimeout.ms=100000
> > > groupid=XXX
> > > autooffset.reset=smallest
> > > socket.buffersize=1048576
> > > fetch.size=10485760
> > > queuedchunks.max=10000
> > >
> > > Thanks for any assistance you can provide,
> > >
> > > Andrew
> >
>

 
+
Andrew Neilson 2013-04-23, 04:13
+
Andrew Neilson 2013-04-23, 04:15
+
Jun Rao 2013-04-23, 14:53
+
Andrew Neilson 2013-04-26, 22:00
+
Neha Narkhede 2013-04-27, 01:36
+
Andrew Neilson 2013-04-27, 01:50