Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Plz help me


Copy link to this message
-
Re: Plz help me
This is a known bug in Kafka 0.7.x (see the discussion in
http://apache.markmail.org/thread/c7tdalfketpusqkg<https://email.corp.linkedin.com/owa/redir.aspx?C=1xDonf6R-kypHyEIBc8--nyHJD81WtAIhCXjIAupqyQiHo7rzRvWm0MQCXLVNb8AR7q88lcHzbI.&URL=http%3a%2f%2fapache.markmail.org%2fthread%2fc7tdalfketpusqkg>
).
Basically, for a new topic, we bootstrap using all existing brokers.
However, if a topic already exists on some brokers, we never bootstrap
again, which means new brokers will be ignored. For now, you have to
manually create the topic on the new brokers (e.g., by sending some data to
the new broker directly).

Thanks,

Jun
On Tue, Jul 16, 2013 at 12:06 AM, Xiaoliang Tian
<[EMAIL PROTECTED]>wrote:

> I have 2 kafka and 1 zookeeper
>
> But When I use kafka producer to send msg to zookeeper ,It just send msg to
> broker 0,but not broker 1.
>
> ./kafka/bin/kafka-producer-perf-test.sh --brokerinfo  zk.connect=
> 211.43.180.171:2181 --messages 100000 --topic my_topic --threads 2
> --message-size 200 --async=true
>
>
> *2013-07-16 15:59:55,158] INFO Creating async producer for broker id = 1 at
> 10.10.6.201:9092(kafka.producer.ProducerPool)
> [2013-07-16 15:59:55,158] INFO Creating async producer for broker id = 0 at
> 10.10.7.166:9092(kafka.producer.ProducerPool)*
> [2013-07-16 15:59:55,159] INFO Producer properties = {zk.connect=
> 211.43.180.171:2181, compression.codec=0, buffer.size=65536,
> reconnect.interval=2147483647, batch.size=200, producer.type=async,
> queue.enqueueTimeout.ms <http://queue.enqueuetimeout.ms/>=-1}
> (kafka.tools.
> ProducerPerformance$AsyncProducerThread)
> [2013-07-16 15:59:55,159] INFO Messages per thread = 50000 (kafka.tools.
> ProducerPerformance$AsyncProducerThread)
> [2013-07-16 15:59:55,159] INFO Initiating client connection, connectString=
> 211.43.180.171:2181sessionTimeout=6000 watcher=org.I0Itec.zkclient.
> ZkClient@17bc72c0 (org.apache.zookeeper.ZooKeeper)
> [2013-07-16 15:59:55,160] INFO Starting ZkClient event thread.
> (org.I0Itec.zkclient.ZkEventThread)
> [2013-07-16 15:59:55,167] INFO Opening socket connection to server /
> 211.43.180.171:2181(org.apache.zookeeper.ClientCnxn)
> *[2013-07-16 15:59:55,417] INFO Connected to 10.10.7.166:9092 for
> producing
> (kafka.producer.SyncProducer)*
> [2013-07-16 15:59:56,656] INFO thread 0: 5000 messages sent 3351.2064
> nMsg/sec 2.1521 MBs/sec (kafka.tools.ProducerPerformance$
> AsyncProducerThread)
> [2013-07-16 15:59:57,323] INFO thread 0: 10000 messages sent 7440.4762
> nMsg/sec 4.8249 MBs/sec (kafka.tools.ProducerPerformance$
> AsyncProducerThread)
> [2013-07-16 15:59:57,774] INFO thread 0: 15000 messages sent 11086.4745
> nMsg/sec 7.1401 MBs/sec (kafka.tools.ProducerPerformance$
> AsyncProducerThread)
> [2013-07-16 15:59:57,970] INFO thread 0: 20000 messages sent 25510.2041
> nMsg/sec 16.3751 MBs/sec (kafka.tools.ProducerPerformance$
> AsyncProducerThread)
> [2013-07-16 15:59:58,101] INFO thread 0: 25000 messages sent 38167.9389
> nMsg/sec 24.7269 MBs/sec (kafka.tools.ProducerPerformance$
> AsyncProducerThread)
> [2013-07-16 15:59:58,250] INFO thread 0: 30000 messages sent 33557.0470
> nMsg/sec 21.6650 MBs/sec (kafka.tools.ProducerPerformance$
> AsyncProducerThread)
> [2013-07-16 15:59:58,395] INFO thread 0: 35000 messages sent 34722.2222
> nMsg/sec 22.4402 MBs/sec (kafka.tools.ProducerPerformance$
> AsyncProducerThread)
> [2013-07-16 15:59:58,530] INFO thread 0: 40000 messages sent 36764.7059
> nMsg/sec 23.6089 MBs/sec (kafka.tools.ProducerPerformance$
> AsyncProducerThread)
> [2013-07-16 15:59:58,687] INFO thread 0: 45000 messages sent 31847.1338
> nMsg/sec 20.4403 MBs/sec (kafka.tools.ProducerPerformance$
> AsyncProducerThread)
> [2013-07-16 15:59:58,797] INFO thread 0: 50000 messages sent 45454.5455
> nMsg/sec 29.1881 MBs/sec (kafka.tools.ProducerPerformance$
> AsyncProducerThread)
> [2013-07-16 15:59:58,797] INFO Closing all async producers
> (kafka.producer.ProducerPool)
>
> producers always connect to broker 0,I don't know why
>
> Plz help me.

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB