Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # dev >> FlumeNG & Twitter Storm


Copy link to this message
-
Re: FlumeNG & Twitter Storm
Hi,

Perhaps an Interceptor [1] may be more appropriate to do this integration.

[1] http://flume.apache.org/FlumeUserGuide.html#flume-interceptors

Regards,
Arvind Prabhakar

On Fri, Aug 17, 2012 at 12:53 AM, Mac Fang <[EMAIL PROTECTED]> wrote:

> Hi, Team,
>
> I am trying to combine the Twitter Storm with FlumeNG to achieve the
> realtime analytics ability. However, it seems Twitter Storm is appropriate
> for 'Queue' based design, since it utilize the 'pull' design to fetch data
> and to keep the message reliability. Whereas Sink of FlumeNG is 'pull' data
> from 'Channel' and 'push' to others. It seems to be conflicted. Are there
> any best practices or any plan for FlumeNG to support Continuous Analytics
> or Realtime Analytics. Or I am trying to imlement a configured Spout in
> Storm to act as a Sink in Flume, but that looks weird.
>
> Many thanks.
>
>
> regards
> macf
>