Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Re: configuring kafka


Copy link to this message
-
Re: configuring kafka
1. Take a look at #6 in http://kafka.apache.org/faq.html

2. Typically, you want each batch to have 10s-100s KB.

3. In the latest 0.8, we have 3 network threads and 8 io threads. These
should be good enough for typical users.

Thanks,

Jun
On Thu, Apr 25, 2013 at 2:30 PM, Yu, Libo <[EMAIL PROTECTED]> wrote:

> Hi,
>
> I have a few questions about configuring kafka 0.8.
> Assume I have 3 brokers, 10 consumers and 4 topics.
>
> 1         How many partitions should a topic have for best performance?
>
> Each broker has one partition? Or each producer should have a partition?
>
> 2         Batch size has big impact on performance, how to determine it
> optimally?
>
> 3         For multiple producers, do I need to increase
> num.network.threads and
>
> num.io.threads from the default of 2?
>
>
> Thanks.
>
> Regards,
>
> Libo
>
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB