Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> Is Avro right for me?


+
Mark 2013-05-23, 20:29
+
Sean Busbey 2013-05-24, 04:16
Copy link to this message
-
Re: Is Avro right for me?
Yes our central server would be Hadoop.

Exactly how would this work with flume? Would I write Avro to a file source which flume would then ship over to one of our collectors  or is there a better/native way? Would I have to include the schema in each event? FYI we would be doing this primarily from a rails application.

Does anyone ever use Avro with a message bus like RabbitMQ?

On May 23, 2013, at 9:16 PM, Sean Busbey <[EMAIL PROTECTED]> wrote:

> Yep. Avro would be great at that (provided your central consumer is Avro friendly, like a Hadoop system).  Make sure that all of your schemas have default values defined for fields so that schema evolution will be easier in the future.
>
>
> On Thu, May 23, 2013 at 4:29 PM, Mark <[EMAIL PROTECTED]> wrote:
> We're thinking about generating logs and events with Avro and shipping them to a central collector service via Flume. Is this a valid use case?
>
>
>
>
> --
> Sean

+
Martin Kleppmann 2013-05-27, 13:25
+
Russell Jurney 2013-05-27, 18:08
+
Stefan Krawczyk 2013-05-27, 19:00
+
Martin Kleppmann 2013-05-27, 19:34
+
Mark 2013-05-28, 22:38
+
Martin Kleppmann 2013-05-29, 10:16
+
Mike Percy 2013-05-29, 00:02
+
Mark 2013-05-29, 16:30
+
Mike Percy 2013-05-30, 03:02
+
Mark 2013-06-05, 03:10
+
Felix GV 2013-06-06, 18:51
+
Felix GV 2013-06-06, 19:09
+
Mark 2013-06-04, 19:57
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB