Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> Is Avro right for me?


+
Mark 2013-05-23, 20:29
+
Sean Busbey 2013-05-24, 04:16
Copy link to this message
-
Re: Is Avro right for me?
Yes our central server would be Hadoop.

Exactly how would this work with flume? Would I write Avro to a file source which flume would then ship over to one of our collectors  or is there a better/native way? Would I have to include the schema in each event? FYI we would be doing this primarily from a rails application.

Does anyone ever use Avro with a message bus like RabbitMQ?

On May 23, 2013, at 9:16 PM, Sean Busbey <[EMAIL PROTECTED]> wrote:

> Yep. Avro would be great at that (provided your central consumer is Avro friendly, like a Hadoop system).  Make sure that all of your schemas have default values defined for fields so that schema evolution will be easier in the future.
>
>
> On Thu, May 23, 2013 at 4:29 PM, Mark <[EMAIL PROTECTED]> wrote:
> We're thinking about generating logs and events with Avro and shipping them to a central collector service via Flume. Is this a valid use case?
>
>
>
>
> --
> Sean

+
Martin Kleppmann 2013-05-27, 13:25
+
Russell Jurney 2013-05-27, 18:08
+
Stefan Krawczyk 2013-05-27, 19:00
+
Martin Kleppmann 2013-05-27, 19:34
+
Mark 2013-05-28, 22:38
+
Martin Kleppmann 2013-05-29, 10:16
+
Mike Percy 2013-05-29, 00:02
+
Mark 2013-05-29, 16:30
+
Mike Percy 2013-05-30, 03:02
+
Mark 2013-06-05, 03:10
+
Felix GV 2013-06-06, 18:51
+
Felix GV 2013-06-06, 19:09
+
Mark 2013-06-04, 19:57