Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka, mail # user - Response time degradation


+
Danny Yeshurun 2013-01-28, 14:02
Copy link to this message
-
Re: Response time degradation
Jun Rao 2013-01-28, 16:32
Not sure what's the causing the slow down. Is the second fetch geting more
data?

Also, if all those partitions are on the same broker, you could have
fetched all partitions' data in a single fetch request.

Thanks,

Jun

On Mon, Jan 28, 2013 at 6:02 AM, Danny Yeshurun <[EMAIL PROTECTED]>wrote:

> Hi,
>
> I am using Kafka 0.7.2
> I have a consumer that tries to fetch messages from several partitions on
> the same broker (and topic).
> The consumers issues a fetch request per partition, if there are messages
> to read, it fetches the first one and goes to the next partition to fetch
> another message and so on (in a loop).
> When there are not messages in a partition, the consumer will issue another
> fetch request on the next cycle etc.
> I noticed that starting with the 2nd fetch request, response time becomes
> unacceptable, i.e. reading back the response from the broker takes ~0.04
> second (while it took ~0.0006 sec on the first time).
> This behavior is consistent over different runs.
>
> Any idea?
>
> Thanks,
> Danny
>

 
+
Danny Yeshurun 2013-01-28, 16:57
+
Neha Narkhede 2013-01-28, 17:54
+
Danny Yeshurun 2013-01-29, 10:41
+
Neha Narkhede 2013-01-29, 16:31
+
Danny Yeshurun 2013-01-30, 16:40