Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume, mail # dev - Review Request: FLUME-1502: Support for running simple configurations embedded in host process


+
Brock Noland 2012-11-19, 19:42
Copy link to this message
-
Re: Review Request: FLUME-1502: Support for running simple configurations embedded in host process
Mike Percy 2012-12-09, 00:24

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8127/#review14085
-----------------------------------------------------------
Brock, looks very nice overall.

Some general suggestions:
1. Can we try to add a Javadoc at the top of every interface and class? It helps to specifies its purpose for other devs / reviewers.
2. Does the EmbeddedAgentConfiguration support making the channel "optional" via a channel selector? I can't tell. If not, can we add support for that?
3. Needs a doc section in the Flume developer guide

Nit: a few whitespace red things on RB

Other specific feedback inline.

flume-ng-core/src/main/java/org/apache/flume/source/AvroSource.java
<https://reviews.apache.org/r/8127/#comment30140>

    Why add these public methods to AvroSource?

flume-ng-embedded-agent/src/main/java/org/apache/flume/agent/embedded/EmbeddedAgent.java
<https://reviews.apache.org/r/8127/#comment30286>

    Should we add
    
      @InterfaceAudience.Public
      @InterfaceStability.Stable
    
    to this constructor?

flume-ng-embedded-agent/src/main/java/org/apache/flume/agent/embedded/EmbeddedAgent.java
<https://reviews.apache.org/r/8127/#comment30341>

    Let's consider only allowing the LocalSource.

flume-ng-embedded-agent/src/main/java/org/apache/flume/agent/embedded/EmbeddedAgent.java
<https://reviews.apache.org/r/8127/#comment30340>

    Should be "Expected one sink group", not one sink, right?

flume-ng-embedded-agent/src/main/java/org/apache/flume/agent/embedded/EmbeddedAgentConfiguration.java
<https://reviews.apache.org/r/8127/#comment30345>

    Can we split this up into validate() and configure() or something... this method does more than simply validate the config.

flume-ng-embedded-agent/src/main/java/org/apache/flume/agent/embedded/MaterializedConfigurationProvider.java
<https://reviews.apache.org/r/8127/#comment30346>

    What does "Materialized" mean? Would be nice to explain the terminology since I find it confusing, or rename the class maybe :)

flume-ng-embedded-agent/src/main/java/org/apache/flume/agent/embedded/package-info.java
<https://reviews.apache.org/r/8127/#comment30347>

    Needs an Apache license header
- Mike Percy
On Nov. 19, 2012, 7:42 p.m., Brock Noland wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/8127/
> -----------------------------------------------------------
>
> (Updated Nov. 19, 2012, 7:42 p.m.)
>
>
> Review request for Flume.
>
>
> Description
> -------
>
> Patch is on top of the FLUME-1502 branch which has 1630 committed to it.
>
> This patch creates a module flume-ng-embedded-agent which implements FLUME-1502. The patch is not read for commit. Specifically, it needs the following:
>
> 1) whitespace removal
> 2) docs in package-info.java and the developer guide
>
> However, I'd like to start soliciting feedback as these are not items which will affect it's overall design.
>
>
> This addresses bug FLUME-1502.
>     https://issues.apache.org/jira/browse/FLUME-1502
>
>
> Diffs
> -----
>
>   flume-ng-configuration/src/main/java/org/apache/flume/conf/FlumeConfiguration.java 9b209e8
>   flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java f39b919
>   flume-ng-core/src/main/java/org/apache/flume/source/AvroSource.java 47ccf9f
>   flume-ng-dist/pom.xml d929c88
>   flume-ng-embedded-agent/pom.xml PRE-CREATION
>   flume-ng-embedded-agent/src/main/java/org/apache/flume/agent/embedded/AvroLocalSource.java PRE-CREATION
>   flume-ng-embedded-agent/src/main/java/org/apache/flume/agent/embedded/EmbeddedAgent.java PRE-CREATION
>   flume-ng-embedded-agent/src/main/java/org/apache/flume/agent/embedded/EmbeddedAgentConfiguration.java PRE-CREATION
>   flume-ng-embedded-agent/src/main/java/org/apache/flume/agent/embedded/EmbeddedSource.java PRE-CREATION
+
Brock Noland 2012-12-09, 17:47
+
Brock Noland 2012-12-10, 16:15
+
Brock Noland 2012-12-10, 19:15
+
Mike Percy 2012-12-11, 00:28
+
Brock Noland 2012-12-11, 00:32
+
Brock Noland 2012-12-10, 20:36
+
Mike Percy 2012-12-11, 20:06
+
Brock Noland 2012-12-11, 21:03
+
Brock Noland 2012-12-11, 21:07
+
Mike Percy 2012-12-12, 00:32
+
Mike Percy 2012-12-12, 00:38