Hi, I am a newbie, I was just checking if there has been any updates on the REST proxy. I have tried to come up with a draft of REST API and can post it for review and discussion if it is still an open topic.
I think this is really cool and would be useful for a lot of people. David, do you think it would really benefit from being a contrib package in kafka versus a separately hosted github project?
Here was our experience: contrib packages tend to get added but the authors and don't necessarily want involvement in kafka internals or necessarily follow the mailing list to answer questions. When we take then they end up kind of stagnating because there is overhead for the code review process, and the main kafka committers aren't generally qualified to fully maintain the code.
What we have seen is that external projects that host add ons do really well. The author tends to get more credit for their work, directly responds to questions from users, etc.
On the other hand maybe we feel this is central enough that it should just be a top-level package in kafka. I think this makes sense for things which are closely tied to internals, are general enough that there is only one good way to do it, and will be very common for kafka users to want. If this sounds better for the REST proxy the important thing is just that someone has bandwidth to be able to support it and ensure we we document it, have sufficient tests to keep it in sync, etc.
Let me know what you think, if the preference is to add it to kafka then I can help with code reviews and stuff.
-Jay On Thu, Jul 18, 2013 at 8:15 PM, David Arthur <[EMAIL PROTECTED]> wrote:
I think there would be many advantages to having a REST API in Kafka, as a core component. Not just for data services (producing/consuming messages), but also for admin. I could imagine a Kafka web console that showed various stats, allowed for maintenance operations, showed pretty viz of the cluster, etc.
The REST prototype I put together using Jetty was really a simplistic POC. In reality, more modern web techniques like chunked transfer encoding and streaming should be used for high throughput (in, and out). Netty would probably be a good choice for a networking library.
I'd be willing to bet that a sufficiently high performance and easy to use REST API could put a lot of the 3rd party client libraries out of business, so-to-speak.
To more directly answer your question, Jay, I think if something like this is done it should be done as a top-level component in Kafka. Is contrib even maintained/updated anymore? Hasn't Camus supplanted the hadoop stuff?
-Jay On Fri, Jul 19, 2013 at 5:25 PM, David Arthur <[EMAIL PROTECTED]> wrote:
Jay Kreps 2013-07-20, 04:48
NEW: Monitor These Apps!
Apache Lucene, Apache Solr and all other Apache Software Foundation projects and their respective logos are trademarks of the Apache Software Foundation.
Elasticsearch, Kibana, Logstash, and Beats are trademarks of Elasticsearch BV, registered in the U.S. and in other countries. This site and Sematext Group is in no way affiliated with Elasticsearch BV.
Service operated by Sematext