Hi Philip. Does it mean to store the same data twice - each time to different partition? I tried to save data only one time. Using two partitions means to store data twice? By the way I am using kafka 0.7.2.
Thanks Oleg. On Sun, Apr 21, 2013 at 11:30 AM, Philip O'Toole <[EMAIL PROTECTED]> wrote:
On Sun, Apr 21, 2013 at 8:53 AM, Oleg Ruchovets <[EMAIL PROTECTED]> wrote:
No, I mean spreading the data across the two partitions, so 50% goes in one, and 50% goes in the other. Have your Producer always write to partition "-1", which will tell Kafka to select a partition at random for each message.
Then one of the Consumers will consume partition 0, the other partition 1.
OK, if you want each consumer to process the same data, then simply point each consumer at your Kafka cluster and have each Consumer consume all data. There is no synchronization required between those two consumers.
In other words, what you want to do is fine. Please read the Kafka design doc if you have not done so:
Hi. I have simple kafka producer/consumer application. I have one producer and 2 consumers. consumers has the same code , it is just executed it in different threads. For some reason information produced by producer consumed only by ONE CONSUMER.Second consumer didn't consumed any information. May be I have to add additional configuration parameters?
I need is the following: 1)Producer produce one event. 2)Two consumers and each consumer consumes the event one time.
For example : 1)Producer1 produces message Message1. 2)Consumer1 consumes Message1. 3)Consumer2 consumes Message1.
What is the way to got such functionality?
Thanks Oleg. On Sun, Apr 21, 2013 at 7:21 PM, Philip O'Toole <[EMAIL PROTECTED]> wrote:
If using the high level consumer you'll need to run two different Groups to have both of your consumers receive every message. I'm assuming you want two different types of business logic to process each message? If so you need to treat them as separate groups. On Fri, Apr 26, 2013 at 11:28 AM, Jun Rao <[EMAIL PROTECTED]> wrote:
Thank you Chris. This is exactly what I tried to achieve. What is the best source to learn Kafka. Actually my question was a very basic and I didn't find any examples or detailed documentation. Is there any book or good documentation ?
Thanks Oleg. On Fri, Apr 26, 2013 at 6:34 PM, Chris Curtin <[EMAIL PROTECTED]>wrote:
Apache Lucene, Apache Solr and all other Apache Software Foundation project and their respective logos are trademarks of the Apache Software Foundation.
Elasticsearch, Kibana, Logstash, and Beats are trademarks of Elasticsearch BV, registered in the U.S. and in other countries. This site and Sematext Group is in no way affiliated with Elasticsearch BV.
Service operated by Sematext