Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Zookeeper >> mail # user >> Zookeeper in Kafka has gets a very low Performance


+
sun liwei 2012-09-12, 10:07
+
sun liwei 2012-09-12, 10:33
Copy link to this message
-
Re: Zookeeper in Kafka has gets a very low Performance
Hi Liwei,

Could JVM warm-up account for the additional time required to getData?

Or do you have log4j logging set to something other than INFO?
DEBUG/TRACE can slow things down.

Regards, Kathleen
On Wed, Sep 12, 2012 at 3:33 AM, sun liwei <[EMAIL PROTECTED]> wrote:
> The version of zookeeper is 3.3.4.
>
> On Wed, Sep 12, 2012 at 6:07 PM, sun liwei <[EMAIL PROTECTED]> wrote:
>
>> Here is what I have:
>>
>>
>>    - zookeeper cluster with 3 severs
>>    - 1700+ kafka topics (that means there are 1700+ children under path
>>    '/brokers/topics'), these topics are written frequently by kafka.
>>    - log file size is 67108880 byte,  snapshot file size is 4192072
>>    byte, new log/snapshot files are created every one or two minutes
>>
>> The problem is that it takes more than 40ms to getData of a single topic.
>> I have another zookeeper cluster and the getData takes no more than 2ms to
>> getData of a single path. This cluster doesn't server kafka.
>>
>> Any body knows why the performance is so bad?
>>
>> Many thanks and best regards.
>>
>> Liwei
>>
+
sun liwei 2012-09-13, 01:20
+
sun liwei 2012-09-13, 01:44
+
Alexander Shraer 2012-09-13, 05:07
+
sun liwei 2012-09-13, 06:47
+
Alexander Shraer 2012-09-13, 13:44
+
Alexander Shraer 2012-09-13, 14:08
+
Kathleen Ting 2012-09-13, 17:21
+
Henry Robinson 2012-09-12, 18:49
+
sun liwei 2012-09-13, 01:24
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB