Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # dev >> FlumeNG & Twitter Storm


+
Mac Fang 2012-08-17, 07:53
+
Arvind Prabhakar 2012-08-17, 17:06
Copy link to this message
-
Re: FlumeNG & Twitter Storm
There is a brief JIRA about this already, would be a good place for
further discussion.

https://issues.apache.org/jira/browse/FLUME-1286

- Patrick

On Fri, Aug 17, 2012 at 10:06 AM, Arvind Prabhakar <[EMAIL PROTECTED]> wrote:
> Hi,
>
> Perhaps an Interceptor [1] may be more appropriate to do this integration.
>
> [1] http://flume.apache.org/FlumeUserGuide.html#flume-interceptors
>
> Regards,
> Arvind Prabhakar
>
> On Fri, Aug 17, 2012 at 12:53 AM, Mac Fang <[EMAIL PROTECTED]> wrote:
>
>> Hi, Team,
>>
>> I am trying to combine the Twitter Storm with FlumeNG to achieve the
>> realtime analytics ability. However, it seems Twitter Storm is appropriate
>> for 'Queue' based design, since it utilize the 'pull' design to fetch data
>> and to keep the message reliability. Whereas Sink of FlumeNG is 'pull' data
>> from 'Channel' and 'push' to others. It seems to be conflicted. Are there
>> any best practices or any plan for FlumeNG to support Continuous Analytics
>> or Realtime Analytics. Or I am trying to imlement a configured Spout in
>> Storm to act as a Sink in Flume, but that looks weird.
>>
>> Many thanks.
>>
>>
>> regards
>> macf
>>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB