Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume, mail # user - Sinking data to a Node.js server/listener


Copy link to this message
-
Sinking data to a Node.js server/listener
Erik Bertrand 2013-02-17, 04:50
I'm looking to get Flume data transferred to a Node.js server listening on
a specific port.  I'm not interested in actually storing the data anywhere,
as it's simply using that data to display "current activity" and doesn't
need to be persist it anywhere.  I'm transferring a very small amount of
data - just a series of IP addresses, actually.  So I'd like to keep the
architecture simple, too.

At first I was thinking I could use the Avro sink to send the data directly
to Node.js using a dnode <https://npmjs.org/package/dnode> server object
(i.e. RPC), but there seems to be more to it than the basic setup.  I'm
just not sure how to configure the Node.js side to understand the Avro sink
RPC request (or if that's even possible).

I've been looking at creating a custom sink to do this; I've not written
one before, much less written anything in Java, so that'd be new to me.
 Any pointers?

Erik
+
Bertrand Dechoux 2013-02-17, 22:43
+
Erik Bertrand 2013-02-21, 13:41
+
Bertrand Dechoux 2013-02-21, 14:08