Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> How to get topic list from kafka server


Copy link to this message
-
Re: How to get topic list from kafka server
I'm curious the use case, but I think you'll need to use a combination of
ZooKeeper meta data and the TopicMetadata API.

The list of known topics for the cluster is in ZooKeeper:
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+data+structures+in+Zookeeper

Also in ZooKeeper is the list of Brokers.

To protect yourself from changes on how Kafka uses ZooKeeper I'd do the
following:
- get the list of topics from ZooKeeper
- get the list of Brokers from ZooKeeper
- connect to the first Broker and use the list of topics to send
TopicMetadataRequest requests to figure out which partitions for the topic
are on which Broker (leader is what you want to know I think)

Since the leader assignments may change, be prepared for 'where' a
topic/partition lives to change. For example the Broker you are talking too
may suddenly be the leader for another topic/partition if that
topic/partition's leader failed and the current Broker was elected leader.
Depending on why you need to know about every topic/partition on a Broker
may have to listen on ZooKeeper events to see that this transition happened.

Hope this helps,
Chris
On Wed, Mar 13, 2013 at 3:38 AM, Snehalata Nagaje <
[EMAIL PROTECTED]> wrote:

> Hi,
>
>
>
> I am using kafka 0.8 version.
>
>
>
> I want to know the list of topics created in kafka server along with it's
> metadata.
>
>
>
> Is there any API available to find out this?
>
>
>
> There is API to fetch TopicMetadata, but this needs name of topic as input
> parameters.
>
>
>
> I need information for all topics present in server.
>
>
>
> Thanks,
>
> Snehalata
>
>
>
>
> Disclaimer:
> This e-mail may contain Privileged/Confidential information and is
> intended only for the individual(s) named. Any review, retransmission,
> dissemination or other use of, or taking of any action in reliance upon
> this information by persons or entities other than the intended recipient
> is prohibited. Please notify the sender, if you have received this e-mail
> by mistake and delete it from your system. Information in this message that
> does not relate to the official business of the company shall be understood
> as neither given nor endorsed by it. E-mail transmission cannot be
> guaranteed to be secure or error-free. The sender does not accept liability
> for any errors or omissions in the contents of this message which arise as
> a result of e-mail transmission. If verification is required please request
> a hard-copy version. Visit us at http://www.harbingergroup.com/
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB