Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> HTTPSource is not listening on the port


Copy link to this message
-
HTTPSource is not listening on the port
Hi all,

I set the flume agent with a HTTPSource , but after I start the agent , the
HTTPSource seems not works properly.
Here is my configuration file:

# The configuration file needs to define the sources,
# the channels and the sinks.
# Sources, channels and sinks are defined per agent,
# in this case called 'agent'

agent.sources = httpSrc
agent.channels = memoryChannel
agent.sinks = loggerSink

# For each one of the sources, the type is defined
agent.sources.httpSrc.type = org.apache.flume.source.http.HTTPSource
agent.sources.httpSrc.port = 51400
agent.sources.httpSrc.handler = org.apache.flume.http.JSONHandler

# The channel can be defined as follows.
agent.sources.httpSrc.channels = memoryChannel

# Each sink's type must be defined
agent.sinks.loggerSink.type = logger

#Specify the channel the sink should use
agent.sinks.loggerSink.channel = memoryChannel

# Each channel's type is defined.
agent.channels.memoryChannel.type = memory

# Other config values specific to each type of channel(sink or source)
# can be defined as well
# In this case, it specifies the capacity of the memory channel
agent.channels.memoryChannel.capacity = 100

Here is the log after starting:

Info: Sourcing environment configuration script
/home/extend/flume/apache-flume-1.3.0-bin/conf/flume-env.sh
+ exec /home/jdk1.6.0_37/bin/java -Xmx20m -Dflume.root.logger=DEBUG,console
-cp
'/home/extend/flume/apache-flume-1.3.0-bin/conf:/home/extend/flume/apache-flume-1.3.0-bin/lib/*'
-Djava.library.path= org.apache.flume.node.Application -f
conf/flume-conf.properties -n agent1
2012-12-31 19:43:29,514 (main) [INFO -
org.apache.flume.lifecycle.LifecycleSupervisor.start(LifecycleSupervisor.java:67)]
Starting lifecycle supervisor 1
2012-12-31 19:43:29,520 (main) [INFO -
org.apache.flume.node.FlumeNode.start(FlumeNode.java:54)] Flume node
starting - agent1
2012-12-31 19:43:29,525 (lifecycleSupervisor-1-0) [INFO -
org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.start(DefaultLogicalNodeManager.java:203)]
Node manager starting
2012-12-31 19:43:29,529 (lifecycleSupervisor-1-1) [INFO -
org.apache.flume.conf.file.AbstractFileConfigurationProvider.start(AbstractFileConfigurationProvider.java:67)]
Configuration provider starting
2012-12-31 19:43:29,529 (lifecycleSupervisor-1-0) [INFO -
org.apache.flume.lifecycle.LifecycleSupervisor.start(LifecycleSupervisor.java:67)]
Starting lifecycle supervisor 9
2012-12-31 19:43:29,531 (lifecycleSupervisor-1-0) [DEBUG -
org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.start(DefaultLogicalNodeManager.java:207)]
Node manager started
2012-12-31 19:43:29,532 (lifecycleSupervisor-1-1) [DEBUG -
org.apache.flume.conf.file.AbstractFileConfigurationProvider.start(AbstractFileConfigurationProvider.java:86)]
Configuration provider started
2012-12-31 19:43:29,533 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:188)]
Checking file:conf/flume-conf.properties for changes
2012-12-31 19:43:29,534 (conf-file-poller-0) [INFO -
org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:195)]
Reloading configuration file:conf/flume-conf.properties
2012-12-31 19:43:29,542 (conf-file-poller-0) [INFO -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:912)]
Added sinks: loggerSink Agent: agent
2012-12-31 19:43:29,543 (conf-file-poller-0) [INFO -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:998)]
Processing:loggerSink
2012-12-31 19:43:29,543 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:1002)]
Created context for loggerSink: type
2012-12-31 19:43:29,543 (conf-file-poller-0) [INFO -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:998)]
Processing:loggerSink
2012-12-31 19:43:29,544 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.isValid(FlumeConfiguration.java:295)]
Starting validation of configuration for agent: agent,
initial-configuration: AgentConfiguration[agent]
SOURCES: {httpSrc={ parameters:{port=51400, channels=memoryChannel,
type=org.apache.flume.source.http.HTTPSource,
handler=org.apache.flume.http.JSONHandler} }}
CHANNELS: {memoryChannel={ parameters:{capacity=100, type=memory} }}
SINKS: {loggerSink={ parameters:{type=logger, channel=memoryChannel} }}

2012-12-31 19:43:29,551 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.validateChannels(FlumeConfiguration.java:450)]
Created channel memoryChannel
2012-12-31 19:43:29,573 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.validateSinks(FlumeConfiguration.java:655)]
Creating sink: loggerSink using LOGGER
2012-12-31 19:43:29,587 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.isValid(FlumeConfiguration.java:353)]
Post validation configuration for agent
AgentConfiguration created without Configuration stubs for which only basic
syntactical validation was performed[agent]
SOURCES: {httpSrc={ parameters:{port=51400, channels=memoryChannel,
type=org.apache.flume.source.http.HTTPSource,
handler=org.apache.flume.http.JSONHandler} }}
CHANNELS: {memoryChannel={ parameters:{capacity=100, type=memory} }}
AgentConfiguration created with Configuration stubs for which full
validation was performed[agent]
SINKS: {loggerSink=ComponentConfiguration[loggerSink]
  CONFIG:
    CHANNEL:memoryChannel
}

2012-12-31 19:43:29,588 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:117)]
Channels:memoryChannel

2012-12-31 19:43:29,588 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.jav
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB