Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - Re: configuring kafka


Copy link to this message
-
Re: configuring kafka
Jun Rao 2013-04-26, 04:05
1. Take a look at #6 in http://kafka.apache.org/faq.html

2. Typically, you want each batch to have 10s-100s KB.

3. In the latest 0.8, we have 3 network threads and 8 io threads. These
should be good enough for typical users.

Thanks,

Jun
On Thu, Apr 25, 2013 at 2:30 PM, Yu, Libo <[EMAIL PROTECTED]> wrote:

> Hi,
>
> I have a few questions about configuring kafka 0.8.
> Assume I have 3 brokers, 10 consumers and 4 topics.
>
> 1         How many partitions should a topic have for best performance?
>
> Each broker has one partition? Or each producer should have a partition?
>
> 2         Batch size has big impact on performance, how to determine it
> optimally?
>
> 3         For multiple producers, do I need to increase
> num.network.threads and
>
> num.io.threads from the default of 2?
>
>
> Thanks.
>
> Regards,
>
> Libo
>
>