Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # dev >> Possible Conflicting Information Regarding Relationship Between Channels and Sinks within Documentation


Copy link to this message
-
Possible Conflicting Information Regarding Relationship Between Channels and Sinks within Documentation
Hey guys,

I have a quick question that I would like to ask based on what I found
within the user and developer documentation.

This could cause some confusion for first time folks.

*Background:*

>From the documentation, a Flume source accepts event data and sends it into
a channel.

These event data are queued up in the channel.

 A sink takes data from the channel for processing (forwarding to another
agent's source or central repo).

Furthermore, *there can be one source, one or more channels, and one or
more sinks for each agent.  *

Within an agent, *a flume source can write to multiple channels, but a sink
can pull events from only one channel.*

Hence, within this context, the relationship between a source and channel
could be one to many but the relationship between a sink and channel is
always one-to-one.

*
Potential Conflicting Information in Documentation*:

On this page, http://flume.apache.org/FlumeUserGuide.html#defining-the-flow

It states that *"A source instance can specify multiple channels, but a
sink instance can only specify one channel."*
However, on this page, http://flume.apache.org/FlumeDeveloperGuide.html#sink

I noticed the following sentence:

*A Sink is associated with one or more Channels, as configured in the Flume
properties file.*
*Question and Next Steps*:

Within what context is this an accurate statement for a sink instance?

>From the context of a single agent, is this an accurate statement? If not
can I create a JIRA issue and submit a patch to correct it?
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB