Kafka, mail # user - Re: Kafka crashed after multiple topics were added - 2013-08-14, 16:27
 Search Hadoop and all its subprojects:

Switch to Plain View
+
Vadim Keylis 2013-08-14, 06:04
+
Jun Rao 2013-08-14, 14:39
+
Vadim Keylis 2013-08-14, 15:20
+
Vadim Keylis 2013-08-14, 16:01
Copy link to this message
-
Re: Kafka crashed after multiple topics were added
We use 30k as the limit. It is largely driven by the number of partitions
(including replicas), retention period and number of
simultaneous producers/consumers.

In your case it seems you have 150 topics, 36 partitions, 3x replication -
with that configuration you will definitely need to up your file handle
limit.

Thanks,

Joel

On Wednesday, August 14, 2013, Vadim Keylis wrote:
 
+
Vadim Keylis 2013-08-14, 16:47
+
Vadim Keylis 2013-08-14, 17:32
+
Joel Koshy 2013-08-14, 20:59
+
Vadim Keylis 2013-08-15, 18:54
+
Jay Kreps 2013-08-15, 20:59
+
Vadim Keylis 2013-08-15, 22:41
+
Jay Kreps 2013-08-15, 23:08
+
Vadim Keylis 2013-08-15, 23:38
+
Jun Rao 2013-08-16, 04:04
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB