Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper >> mail # user >> Zookeeper in Kafka has gets a very low Performance


Copy link to this message
-
Re: Zookeeper in Kafka has gets a very low Performance
On 12 September 2012 03:07, sun liwei <[EMAIL PROTECTED]> wrote:

> Here is what I have:
>
>
>    - zookeeper cluster with 3 severs
>    - 1700+ kafka topics (that means there are 1700+ children under path
>    '/brokers/topics'), these topics are written frequently by kafka.
>    - log file size is 67108880 byte,  snapshot file size is 4192072
>    byte, new log/snapshot files are created every one or two minutes
>
> The problem is that it takes more than 40ms to getData of a single topic. I
> have another zookeeper cluster and the getData takes no more than 2ms to
> getData of a single path. This cluster doesn't server kafka.
>
> Any body knows why the performance is so bad?
>

How big (in terms of bytes) are the topic nodes that you are getting? Are
they the same size as the nodes in the better-performing cluster?

Henry
>
> Many thanks and best regards.
>
> Liwei
>

--
Henry Robinson
Software Engineer
Cloudera
415-994-6679
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB