Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # dev - Re: Review Request 14041: MetadataResponse during Consumer's Rebalance Process maybe Stale


Copy link to this message
-
Re: Review Request 14041: MetadataResponse during Consumer's Rebalance Process maybe Stale
Neha Narkhede 2013-09-09, 23:44

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14041/#review26005
-----------------------------------------------------------

core/src/main/scala/kafka/client/ClientUtils.scala
<https://reviews.apache.org/r/14041/#comment50755>

    The concern I have with this change is the potential performance hit to the consumer rebalance process. Since it affects the end-to-end message latency, I would encourage you to run some performance tests for both a MirrorMaker that consumes ~500 topics as well as a consumer that consumes 5 topics, to quantify the performance hit.
    
    Also, what do you think about the alternate solution that I described on the JIRA?
- Neha Narkhede
On Sept. 9, 2013, 9:31 p.m., Guozhang Wang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14041/
> -----------------------------------------------------------
>
> (Updated Sept. 9, 2013, 9:31 p.m.)
>
>
> Review request for kafka.
>
>
> Bugs: KAFKA-1030
>     https://issues.apache.org/jira/browse/KAFKA-1030
>
>
> Repository: kafka
>
>
> Description
> -------
>
> Using the approach of reading directly from ZK.
>
>
> Diffs
> -----
>
>   core/src/main/scala/kafka/client/ClientUtils.scala cc526ec933052b239f0e7ce43e76cd9d011d5bd9
>   core/src/main/scala/kafka/consumer/ZookeeperConsumerConnector.scala e7a692a1d23ca5a9ecf86e3cb34be418b9c0c943
>
> Diff: https://reviews.apache.org/r/14041/diff/
>
>
> Testing
> -------
>
> unit tests
>
>
> Thanks,
>
> Guozhang Wang
>
>