| clear query|facets|time |
Search criteria: .
Results from 91 to 100 from
120 (0.226s).
|
|
|
Loading phrases to help you
refine your search...
|
|
Keeping track of offsets - Kafka - [mail # user]
|
|
...I'm working on offset management for my Python client (non-ZK). I'm having trouble seeing how you would keep track of the message offset when using compression. As I understand it, when you ...
|
|
|
Author: David Arthur,
2012-11-19, 20:45
|
|
|
Re: On SIGTERM and SIGKILL consumer ids in zookeeper do not always get cleared - Kafka - [mail # user]
|
|
...BTW, this is a feature of the ephemeral nodes in ZooKeeper, not a feature of Kafka. If Kafka doesn't close the ZK connection explicitly, ZK waits for the session timeout before deleting ephe...
|
|
|
Author: David Arthur,
2012-11-19, 19:34
|
|
|
Re: Kafka on EC2 - Kafka - [mail # user]
|
|
...I'd only consider m1.xlarge and higher for Kafka. The m1.xlarge have "high" I/O performance according to Amazon. This is disk I/O and network I/O performance. Of course you need to use EBS v...
|
|
|
Author: David Arthur,
2012-11-19, 19:30
|
|
|
Re: version of zookeeper - Kafka - [mail # user]
|
|
...The ZooKeeper API has been stable for a while, I do not suspect there would be any problems between 3.3.4 and 3.4.x. Personally, I have run Kafka 0.7.x with ZooKeeper 3.4.3 with no problems ...
|
|
|
Author: David Arthur,
2012-11-13, 13:55
|
|
|
Re: Kafka stream processing framework? - Kafka - [mail # dev]
|
|
...Prashanth, Storm seems to be more focused on data flow between Storm processors (spout? bolt? i forget). My particular use case follows this pattern: * read id from kafka queue *...
|
|
|
Author: David Arthur,
2012-11-08, 16:38
|
|
|
Re: Committing offsets - Kafka - [mail # user]
|
|
...Thanks, Jun. This is what I was looking for. I did a little experimentation and found that if I crashed the JVM immediately after consuming a message, the offset is not always changed. I'm g...
|
|
|
Author: David Arthur,
2012-11-08, 16:32
|
|
|
Kafka stream processing framework? - Kafka - [mail # dev]
|
|
...There is a line item on the project ideas for "improved stream processing libraries". I was wondering if anyone has done any work on this. I know you can hook Kafka into things like Storm an...
|
|
|
Author: David Arthur,
2012-11-08, 15:32
|
|
|
Re: Multi Costumer and Broker - Kafka - [mail # dev]
|
|
...Read through the quick start on the project page http://incubator.apache.org/kafka/quickstart.html Kafka consumers need to be in the same "consumer group" in order to act as one (one m...
|
|
|
Author: David Arthur,
2012-11-08, 15:11
|
|
|
Committing offsets - Kafka - [mail # user]
|
|
...Suppose I'm iterating through messages from a topic with 6 messages. I consume message three messages and then commit (or an auto-commit happens). message 1 message 2 message 3 ...
|
|
|
Author: David Arthur,
2012-11-08, 14:22
|
|
|
Re: Scenarios of Hadoop producers and consumers - Kafka - [mail # user]
|
|
...Indeed Hadoop is not the ideal platform for stream processing, but there are plenty of use cases for Kakfa + Hadoop. I use it to consolidate log data from many different systems into HDFS. I...
|
|
|
Author: David Arthur,
2012-10-31, 01:05
|
|
|
|
