Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # dev >> FlumeNG & Twitter Storm


Copy link to this message
-
Re: FlumeNG & Twitter Storm
Hi,

Perhaps an Interceptor [1] may be more appropriate to do this integration.

[1] http://flume.apache.org/FlumeUserGuide.html#flume-interceptors

Regards,
Arvind Prabhakar

On Fri, Aug 17, 2012 at 12:53 AM, Mac Fang <[EMAIL PROTECTED]> wrote:

> Hi, Team,
>
> I am trying to combine the Twitter Storm with FlumeNG to achieve the
> realtime analytics ability. However, it seems Twitter Storm is appropriate
> for 'Queue' based design, since it utilize the 'pull' design to fetch data
> and to keep the message reliability. Whereas Sink of FlumeNG is 'pull' data
> from 'Channel' and 'push' to others. It seems to be conflicted. Are there
> any best practices or any plan for FlumeNG to support Continuous Analytics
> or Realtime Analytics. Or I am trying to imlement a configured Spout in
> Storm to act as a Sink in Flume, but that looks weird.
>
> Many thanks.
>
>
> regards
> macf
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB