Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # dev >> FlumeNG & Twitter Storm

Copy link to this message
Re: FlumeNG & Twitter Storm
There is a brief JIRA about this already, would be a good place for
further discussion.


- Patrick

On Fri, Aug 17, 2012 at 10:06 AM, Arvind Prabhakar <[EMAIL PROTECTED]> wrote:
> Hi,
> Perhaps an Interceptor [1] may be more appropriate to do this integration.
> [1] http://flume.apache.org/FlumeUserGuide.html#flume-interceptors
> Regards,
> Arvind Prabhakar
> On Fri, Aug 17, 2012 at 12:53 AM, Mac Fang <[EMAIL PROTECTED]> wrote:
>> Hi, Team,
>> I am trying to combine the Twitter Storm with FlumeNG to achieve the
>> realtime analytics ability. However, it seems Twitter Storm is appropriate
>> for 'Queue' based design, since it utilize the 'pull' design to fetch data
>> and to keep the message reliability. Whereas Sink of FlumeNG is 'pull' data
>> from 'Channel' and 'push' to others. It seems to be conflicted. Are there
>> any best practices or any plan for FlumeNG to support Continuous Analytics
>> or Realtime Analytics. Or I am trying to imlement a configured Spout in
>> Storm to act as a Sink in Flume, but that looks weird.
>> Many thanks.
>> regards
>> macf