Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> Is Avro right for me?


+
Mark 2013-05-23, 20:29
+
Sean Busbey 2013-05-24, 04:16
+
Mark 2013-05-26, 16:39
+
Martin Kleppmann 2013-05-27, 13:25
+
Russell Jurney 2013-05-27, 18:08
+
Stefan Krawczyk 2013-05-27, 19:00
+
Martin Kleppmann 2013-05-27, 19:34
+
Mark 2013-05-28, 22:38
Copy link to this message
-
Re: Is Avro right for me?
On 28 May 2013, at 23:38, Mark <[EMAIL PROTECTED]> wrote:
> I actually looked into Kafka quite some time ago and I think we passed on it because it didn't have much ruby support (That may have changed by now).

Ruby support unfortunately continues to be weak. I actually wrote a brand new Kafka producer client for Ruby (not yet open source), but from Kafka 0.7 to 0.8 the wire protocol is changing (as replication & high availability features are added), which means I'm going to have to re-do it.

The Kafka project officially supports producers using the JVM or the C client (the latter could be embedded in Ruby using FFI), and consumers using the JVM client only.

However, that doesn't rule out other languages. LinkedIn actually operates a large service that consumes Kafka using the JVM client, writes the Avro messages to stdout as JSON, and then pipes that into a python process' stdin. Works surprisingly well!

Martin
+
Mike Percy 2013-05-29, 00:02
+
Mark 2013-05-29, 16:30
+
Mike Percy 2013-05-30, 03:02
+
Mark 2013-06-05, 03:10
+
Felix GV 2013-06-06, 18:51
+
Felix GV 2013-06-06, 19:09
+
Mark 2013-06-04, 19:57