Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume, mail # user - HTTPSource is not listening on the port


Copy link to this message
-
HTTPSource is not listening on the port
Felix.徐 2012-12-31, 12:07
Hi all,

I set the flume agent with a HTTPSource , but after I start the agent , the
HTTPSource seems not works properly.
Here is my configuration file:

# The configuration file needs to define the sources,
# the channels and the sinks.
# Sources, channels and sinks are defined per agent,
# in this case called 'agent'

agent.sources = httpSrc
agent.channels = memoryChannel
agent.sinks = loggerSink

# For each one of the sources, the type is defined
agent.sources.httpSrc.type = org.apache.flume.source.http.HTTPSource
agent.sources.httpSrc.port = 51400
agent.sources.httpSrc.handler = org.apache.flume.http.JSONHandler

# The channel can be defined as follows.
agent.sources.httpSrc.channels = memoryChannel

# Each sink's type must be defined
agent.sinks.loggerSink.type = logger

#Specify the channel the sink should use
agent.sinks.loggerSink.channel = memoryChannel

# Each channel's type is defined.
agent.channels.memoryChannel.type = memory

# Other config values specific to each type of channel(sink or source)
# can be defined as well
# In this case, it specifies the capacity of the memory channel
agent.channels.memoryChannel.capacity = 100

Here is the log after starting:

Info: Sourcing environment configuration script
/home/extend/flume/apache-flume-1.3.0-bin/conf/flume-env.sh
+ exec /home/jdk1.6.0_37/bin/java -Xmx20m -Dflume.root.logger=DEBUG,console
-cp
'/home/extend/flume/apache-flume-1.3.0-bin/conf:/home/extend/flume/apache-flume-1.3.0-bin/lib/*'
-Djava.library.path= org.apache.flume.node.Application -f
conf/flume-conf.properties -n agent1
2012-12-31 19:43:29,514 (main) [INFO -
org.apache.flume.lifecycle.LifecycleSupervisor.start(LifecycleSupervisor.java:67)]
Starting lifecycle supervisor 1
2012-12-31 19:43:29,520 (main) [INFO -
org.apache.flume.node.FlumeNode.start(FlumeNode.java:54)] Flume node
starting - agent1
2012-12-31 19:43:29,525 (lifecycleSupervisor-1-0) [INFO -
org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.start(DefaultLogicalNodeManager.java:203)]
Node manager starting
2012-12-31 19:43:29,529 (lifecycleSupervisor-1-1) [INFO -
org.apache.flume.conf.file.AbstractFileConfigurationProvider.start(AbstractFileConfigurationProvider.java:67)]
Configuration provider starting
2012-12-31 19:43:29,529 (lifecycleSupervisor-1-0) [INFO -
org.apache.flume.lifecycle.LifecycleSupervisor.start(LifecycleSupervisor.java:67)]
Starting lifecycle supervisor 9
2012-12-31 19:43:29,531 (lifecycleSupervisor-1-0) [DEBUG -
org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.start(DefaultLogicalNodeManager.java:207)]
Node manager started
2012-12-31 19:43:29,532 (lifecycleSupervisor-1-1) [DEBUG -
org.apache.flume.conf.file.AbstractFileConfigurationProvider.start(AbstractFileConfigurationProvider.java:86)]
Configuration provider started
2012-12-31 19:43:29,533 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:188)]
Checking file:conf/flume-conf.properties for changes
2012-12-31 19:43:29,534 (conf-file-poller-0) [INFO -
org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:195)]
Reloading configuration file:conf/flume-conf.properties
2012-12-31 19:43:29,542 (conf-file-poller-0) [INFO -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:912)]
Added sinks: loggerSink Agent: agent
2012-12-31 19:43:29,543 (conf-file-poller-0) [INFO -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:998)]
Processing:loggerSink
2012-12-31 19:43:29,543 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:1002)]
Created context for loggerSink: type
2012-12-31 19:43:29,543 (conf-file-poller-0) [INFO -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:998)]
Processing:loggerSink
2012-12-31 19:43:29,544 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.isValid(FlumeConfiguration.java:295)]
Starting validation of configuration for agent: agent,
initial-configuration: AgentConfiguration[agent]
SOURCES: {httpSrc={ parameters:{port=51400, channels=memoryChannel,
type=org.apache.flume.source.http.HTTPSource,
handler=org.apache.flume.http.JSONHandler} }}
CHANNELS: {memoryChannel={ parameters:{capacity=100, type=memory} }}
SINKS: {loggerSink={ parameters:{type=logger, channel=memoryChannel} }}

2012-12-31 19:43:29,551 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.validateChannels(FlumeConfiguration.java:450)]
Created channel memoryChannel
2012-12-31 19:43:29,573 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.validateSinks(FlumeConfiguration.java:655)]
Creating sink: loggerSink using LOGGER
2012-12-31 19:43:29,587 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.isValid(FlumeConfiguration.java:353)]
Post validation configuration for agent
AgentConfiguration created without Configuration stubs for which only basic
syntactical validation was performed[agent]
SOURCES: {httpSrc={ parameters:{port=51400, channels=memoryChannel,
type=org.apache.flume.source.http.HTTPSource,
handler=org.apache.flume.http.JSONHandler} }}
CHANNELS: {memoryChannel={ parameters:{capacity=100, type=memory} }}
AgentConfiguration created with Configuration stubs for which full
validation was performed[agent]
SINKS: {loggerSink=ComponentConfiguration[loggerSink]
  CONFIG:
    CHANNEL:memoryChannel
}

2012-12-31 19:43:29,588 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:117)]
Channels:memoryChannel

2012-12-31 19:43:29,588 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.jav
+
Hari Shreedharan 2012-12-31, 18:41
+
Felix.徐 2013-01-04, 05:19