Hi all,

I set the flume agent with a HTTPSource , but after I start the agent , the
HTTPSource seems not works properly.
Here is my configuration file:

# The configuration file needs to define the sources,
# the channels and the sinks.
# Sources, channels and sinks are defined per agent,
# in this case called 'agent'

agent.sources = httpSrc
agent.channels = memoryChannel
agent.sinks = loggerSink

# For each one of the sources, the type is defined
agent.sources.httpSrc.type = org.apache.flume.source.http.HTTPSource
agent.sources.httpSrc.port = 51400
agent.sources.httpSrc.handler = org.apache.flume.http.JSONHandler

# The channel can be defined as follows.
agent.sources.httpSrc.channels = memoryChannel

# Each sink's type must be defined
agent.sinks.loggerSink.type = logger

#Specify the channel the sink should use
agent.sinks.loggerSink.channel = memoryChannel

# Each channel's type is defined.
agent.channels.memoryChannel.type = memory

# Other config values specific to each type of channel(sink or source)
# can be defined as well
# In this case, it specifies the capacity of the memory channel
agent.channels.memoryChannel.capacity = 100

Here is the log after starting:

Info: Sourcing environment configuration script
/home/extend/flume/apache-flume-1.3.0-bin/conf/flume-env.sh
+ exec /home/jdk1.6.0_37/bin/java -Xmx20m -Dflume.root.logger=DEBUG,console
-cp
'/home/extend/flume/apache-flume-1.3.0-bin/conf:/home/extend/flume/apache-flume-1.3.0-bin/lib/*'
-Djava.library.path= org.apache.flume.node.Application -f
conf/flume-conf.properties -n agent1
2012-12-31 19:43:29,514 (main) [INFO -
org.apache.flume.lifecycle.LifecycleSupervisor.start(LifecycleSupervisor.java:67)]
Starting lifecycle supervisor 1
2012-12-31 19:43:29,520 (main) [INFO -
org.apache.flume.node.FlumeNode.start(FlumeNode.java:54)] Flume node
starting - agent1
2012-12-31 19:43:29,525 (lifecycleSupervisor-1-0) [INFO -
org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.start(DefaultLogicalNodeManager.java:203)]
Node manager starting
2012-12-31 19:43:29,529 (lifecycleSupervisor-1-1) [INFO -
org.apache.flume.conf.file.AbstractFileConfigurationProvider.start(AbstractFileConfigurationProvider.java:67)]
Configuration provider starting
2012-12-31 19:43:29,529 (lifecycleSupervisor-1-0) [INFO -
org.apache.flume.lifecycle.LifecycleSupervisor.start(LifecycleSupervisor.java:67)]
Starting lifecycle supervisor 9
2012-12-31 19:43:29,531 (lifecycleSupervisor-1-0) [DEBUG -
org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.start(DefaultLogicalNodeManager.java:207)]
Node manager started
2012-12-31 19:43:29,532 (lifecycleSupervisor-1-1) [DEBUG -
org.apache.flume.conf.file.AbstractFileConfigurationProvider.start(AbstractFileConfigurationProvider.java:86)]
Configuration provider started
2012-12-31 19:43:29,533 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:188)]
Checking file:conf/flume-conf.properties for changes
2012-12-31 19:43:29,534 (conf-file-poller-0) [INFO -
org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:195)]
Reloading configuration file:conf/flume-conf.properties
2012-12-31 19:43:29,542 (conf-file-poller-0) [INFO -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:912)]
Added sinks: loggerSink Agent: agent
2012-12-31 19:43:29,543 (conf-file-poller-0) [INFO -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:998)]
Processing:loggerSink
2012-12-31 19:43:29,543 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:1002)]
Created context for loggerSink: type
2012-12-31 19:43:29,543 (conf-file-poller-0) [INFO -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:998)]
Processing:loggerSink
2012-12-31 19:43:29,544 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.isValid(FlumeConfiguration.java:295)]
Starting validation of configuration for agent: agent,
initial-configuration: AgentConfiguration[agent]
SOURCES: {httpSrc={ parameters:{port=51400, channels=memoryChannel,
type=org.apache.flume.source.http.HTTPSource,
handler=org.apache.flume.http.JSONHandler} }}
CHANNELS: {memoryChannel={ parameters:{capacity=100, type=memory} }}
SINKS: {loggerSink={ parameters:{type=logger, channel=memoryChannel} }}

2012-12-31 19:43:29,551 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.validateChannels(FlumeConfiguration.java:450)]
Created channel memoryChannel
2012-12-31 19:43:29,573 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.validateSinks(FlumeConfiguration.java:655)]
Creating sink: loggerSink using LOGGER
2012-12-31 19:43:29,587 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.isValid(FlumeConfiguration.java:353)]
Post validation configuration for agent
AgentConfiguration created without Configuration stubs for which only basic
syntactical validation was performed[agent]
SOURCES: {httpSrc={ parameters:{port=51400, channels=memoryChannel,
type=org.apache.flume.source.http.HTTPSource,
handler=org.apache.flume.http.JSONHandler} }}
CHANNELS: {memoryChannel={ parameters:{capacity=100, type=memory} }}
AgentConfiguration created with Configuration stubs for which full
validation was performed[agent]
SINKS: {loggerSink=ComponentConfiguration[loggerSink]
  CONFIG:
    CHANNEL:memoryChannel
}

2012-12-31 19:43:29,588 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:117)]
Channels:memoryChannel

2012-12-31 19:43:29,588 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.jav

Looks like your config file contains the configuration for an agent named
"agent" and you are trying to run flume with the agent name "agent1". Run
the flume agent as agent or change the name of your agent in the config
file to "agent1" (simply replace all instances of "agent" with "agent1"

Hari

On Monday, December 31, 2012, Felix.徐 wrote:

> Hi all,
>
> I set the flume agent with a HTTPSource , but after I start the agent ,
> the HTTPSource seems not works properly.
> Here is my configuration file:
>
> # The configuration file needs to define the sources,
> # the channels and the sinks.
> # Sources, channels and sinks are defined per agent,
> # in this case called 'agent'
>
> agent.sources = httpSrc
> agent.channels = memoryChannel
> agent.sinks = loggerSink
>
> # For each one of the sources, the type is defined
> agent.sources.httpSrc.type = org.apache.flume.source.http.HTTPSource
> agent.sources.httpSrc.port = 51400
> agent.sources.httpSrc.handler = org.apache.flume.http.JSONHandler
>
> # The channel can be defined as follows.
> agent.sources.httpSrc.channels = memoryChannel
>
> # Each sink's type must be defined
> agent.sinks.loggerSink.type = logger
>
> #Specify the channel the sink should use
> agent.sinks.loggerSink.channel = memoryChannel
>
> # Each channel's type is defined.
> agent.channels.memoryChannel.type = memory
>
> # Other config values specific to each type of channel(sink or source)
> # can be defined as well
> # In this case, it specifies the capacity of the memory channel
> agent.channels.memoryChannel.capacity = 100
>
>
>
> Here is the log after starting:
>
> Info: Sourcing environment configuration script
> /home/extend/flume/apache-flume-1.3.0-bin/conf/flume-env.sh
> + exec /home/jdk1.6.0_37/bin/java -Xmx20m
> -Dflume.root.logger=DEBUG,console -cp
> '/home/extend/flume/apache-flume-1.3.0-bin/conf:/home/extend/flume/apache-flume-1.3.0-bin/lib/*'
> -Djava.library.path= org.apache.flume.node.Application -f
> conf/flume-conf.properties -n agent1
> 2012-12-31 19:43:29,514 (main) [INFO -
> org.apache.flume.lifecycle.LifecycleSupervisor.start(LifecycleSupervisor.java:67)]
> Starting lifecycle supervisor 1
> 2012-12-31 19:43:29,520 (main) [INFO -
> org.apache.flume.node.FlumeNode.start(FlumeNode.java:54)] Flume node
> starting - agent1
> 2012-12-31 19:43:29,525 (lifecycleSupervisor-1-0) [INFO -
> org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.start(DefaultLogicalNodeManager.java:203)]
> Node manager starting
> 2012-12-31 19:43:29,529 (lifecycleSupervisor-1-1) [INFO -
> org.apache.flume.conf.file.AbstractFileConfigurationProvider.start(AbstractFileConfigurationProvider.java:67)]
> Configuration provider starting
> 2012-12-31 19:43:29,529 (lifecycleSupervisor-1-0) [INFO -
> org.apache.flume.lifecycle.LifecycleSupervisor.start(LifecycleSupervisor.java:67)]
> Starting lifecycle supervisor 9
> 2012-12-31 19:43:29,531 (lifecycleSupervisor-1-0) [DEBUG -
> org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.start(DefaultLogicalNodeManager.java:207)]
> Node manager started
> 2012-12-31 19:43:29,532 (lifecycleSupervisor-1-1) [DEBUG -
> org.apache.flume.conf.file.AbstractFileConfigurationProvider.start(AbstractFileConfigurationProvider.java:86)]
> Configuration provider started
> 2012-12-31 19:43:29,533 (conf-file-poller-0) [DEBUG -
> org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:188)]
> Checking file:conf/flume-conf.properties for changes
> 2012-12-31 19:43:29,534 (conf-file-poller-0) [INFO -
> org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:195)]
> Reloading configuration file:conf/flume-conf.properties
> 2012-12-31 19:43:29,542 (conf-file-poller-0) [INFO -
> org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:912)]
> Added sinks: loggerSink Agent: agent
> 2012-12-31 19:43:29,543 (conf-file-poller-0) [INFO -

I see , thanks very much!
2013/1/1 Hari Shreedharan <[EMAIL PROTECTED]>

> Looks like your config file contains the configuration for an agent named
> "agent" and you are trying to run flume with the agent name "agent1". Run
> the flume agent as agent or change the name of your agent in the config
> file to "agent1" (simply replace all instances of "agent" with "agent1"
>
> Hari
>
>
> On Monday, December 31, 2012, Felix.徐 wrote:
>
>> Hi all,
>>
>> I set the flume agent with a HTTPSource , but after I start the agent ,
>> the HTTPSource seems not works properly.
>> Here is my configuration file:
>>
>> # The configuration file needs to define the sources,
>> # the channels and the sinks.
>> # Sources, channels and sinks are defined per agent,
>> # in this case called 'agent'
>>
>> agent.sources = httpSrc
>> agent.channels = memoryChannel
>> agent.sinks = loggerSink
>>
>> # For each one of the sources, the type is defined
>> agent.sources.httpSrc.type = org.apache.flume.source.http.HTTPSource
>> agent.sources.httpSrc.port = 51400
>> agent.sources.httpSrc.handler = org.apache.flume.http.JSONHandler
>>
>> # The channel can be defined as follows.
>> agent.sources.httpSrc.channels = memoryChannel
>>
>> # Each sink's type must be defined
>> agent.sinks.loggerSink.type = logger
>>
>> #Specify the channel the sink should use
>> agent.sinks.loggerSink.channel = memoryChannel
>>
>> # Each channel's type is defined.
>> agent.channels.memoryChannel.type = memory
>>
>> # Other config values specific to each type of channel(sink or source)
>> # can be defined as well
>> # In this case, it specifies the capacity of the memory channel
>> agent.channels.memoryChannel.capacity = 100
>>
>>
>>
>> Here is the log after starting:
>>
>> Info: Sourcing environment configuration script
>> /home/extend/flume/apache-flume-1.3.0-bin/conf/flume-env.sh
>> + exec /home/jdk1.6.0_37/bin/java -Xmx20m
>> -Dflume.root.logger=DEBUG,console -cp
>> '/home/extend/flume/apache-flume-1.3.0-bin/conf:/home/extend/flume/apache-flume-1.3.0-bin/lib/*'
>> -Djava.library.path= org.apache.flume.node.Application -f
>> conf/flume-conf.properties -n agent1
>> 2012-12-31 19:43:29,514 (main) [INFO -
>> org.apache.flume.lifecycle.LifecycleSupervisor.start(LifecycleSupervisor.java:67)]
>> Starting lifecycle supervisor 1
>> 2012-12-31 19:43:29,520 (main) [INFO -
>> org.apache.flume.node.FlumeNode.start(FlumeNode.java:54)] Flume node
>> starting - agent1
>> 2012-12-31 19:43:29,525 (lifecycleSupervisor-1-0) [INFO -
>> org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.start(DefaultLogicalNodeManager.java:203)]
>> Node manager starting
>> 2012-12-31 19:43:29,529 (lifecycleSupervisor-1-1) [INFO -
>> org.apache.flume.conf.file.AbstractFileConfigurationProvider.start(AbstractFileConfigurationProvider.java:67)]
>> Configuration provider starting
>> 2012-12-31 19:43:29,529 (lifecycleSupervisor-1-0) [INFO -
>> org.apache.flume.lifecycle.LifecycleSupervisor.start(LifecycleSupervisor.java:67)]
>> Starting lifecycle supervisor 9
>> 2012-12-31 19:43:29,531 (lifecycleSupervisor-1-0) [DEBUG -
>> org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.start(DefaultLogicalNodeManager.java:207)]
>> Node manager started
>> 2012-12-31 19:43:29,532 (lifecycleSupervisor-1-1) [DEBUG -
>> org.apache.flume.conf.file.AbstractFileConfigurationProvider.start(AbstractFileConfigurationProvider.java:86)]
>> Configuration provider started
>> 2012-12-31 19:43:29,533 (conf-file-poller-0) [DEBUG -
>> org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:188)]
>> Checking file:conf/flume-conf.properties for changes
>> 2012-12-31 19:43:29,534 (conf-file-poller-0) [INFO -
>> org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:195)]
>> Reloading configuration file:conf/flume-conf.properties
>> 2012-12-31 19:43:29,542 (conf-file-poller-0) [INFO -
>> org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:912)]