Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> java, oom, gc stop the world


Copy link to this message
-
java, oom, gc stop the world
In the wild, what sort of memory usage patterns have you guys seen with
kafka?

I'm not that well versed with java and its memory/gc issues, what have you
guys experienced with your production brokers?  (assuming consumers and
producers are not that much of an issue).

How do you determine how much memory to allocate to a given broker?  Is it
based on factors such as:

1. producers are mostly using batch versus single messages
2. messages per second
3. size of message payload
4. # of consumers connecting to a given broker
etc.

Is there a "magic" forumula?

 
+
Jay Kreps 2013-01-23, 03:35
+
David DeMaagd 2013-01-23, 04:24
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB