Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume, mail # dev - FlumeNG & Twitter Storm


Copy link to this message
-
Re: FlumeNG & Twitter Storm
Patrick Wendell 2012-08-17, 20:55
There is a brief JIRA about this already, would be a good place for
further discussion.

https://issues.apache.org/jira/browse/FLUME-1286

- Patrick

On Fri, Aug 17, 2012 at 10:06 AM, Arvind Prabhakar <[EMAIL PROTECTED]> wrote:
> Hi,
>
> Perhaps an Interceptor [1] may be more appropriate to do this integration.
>
> [1] http://flume.apache.org/FlumeUserGuide.html#flume-interceptors
>
> Regards,
> Arvind Prabhakar
>
> On Fri, Aug 17, 2012 at 12:53 AM, Mac Fang <[EMAIL PROTECTED]> wrote:
>
>> Hi, Team,
>>
>> I am trying to combine the Twitter Storm with FlumeNG to achieve the
>> realtime analytics ability. However, it seems Twitter Storm is appropriate
>> for 'Queue' based design, since it utilize the 'pull' design to fetch data
>> and to keep the message reliability. Whereas Sink of FlumeNG is 'pull' data
>> from 'Channel' and 'push' to others. It seems to be conflicted. Are there
>> any best practices or any plan for FlumeNG to support Continuous Analytics
>> or Realtime Analytics. Or I am trying to imlement a configured Spout in
>> Storm to act as a Sink in Flume, but that looks weird.
>>
>> Many thanks.
>>
>>
>> regards
>> macf
>>