Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> Problems with Flume on windows with HTTP source


+
Nickolay Kolev 2013-06-20, 15:57
+
Hari Shreedharan 2013-06-20, 18:07
+
Nickolay Kolev 2013-06-21, 06:34
+
Ashish 2013-06-21, 06:48
+
Nickolay Kolev 2013-06-21, 07:03
+
Ashish 2013-06-21, 07:40
+
Nickolay Kolev 2013-06-21, 08:29
Copy link to this message
-
Re: Problems with Flume on windows with HTTP source
Hi Asish,

I think I found the problem - there must be something wrong with the
configuration and the source is not created. Unfortunately I can not figure
what.
Did you test it with my configuration?

Here is what I found in server log:
2013-06-21 11:01:24,433 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.isValid(FlumeConfiguration.java:295)]
Starting validation of configuration for agent: agent1,
initial-configuration: AgentConfiguration[agent1]
SOURCES: {r1={ parameters:{port=9001, handler.nickname=json handler,
channels=ch1, type=org.apache.flume.source.http.HTTPSource,
handler=org.apache.flume.source.http.JSONHandler} }
}
CHANNELS: {ch1={ parameters:{transactionCapacity=1000, capacity=10000,
type=memory} }}
SINKS: {k1={ parameters:{type=file_roll, channel=ch1,
sink.directory=C:\Projects\nickyk\apache-flume-1.3.1-bin\log} }}

2013-06-21 11:01:24,451 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.validateChannels(FlumeConfiguration.java:450)]
Created channel ch1

2013-06-21 11:01:24,451 (conf-file-poller-0) [WARN -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.validateSources(FlumeConfiguration.java:489)]
Agent configuration for 'agent1' has no sources.
2013-06-21 11:01:24,455 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.validateSinks(FlumeConfiguration.java:655)]
Creating sink: k1 using FILE_ROLL
2013-06-21 11:01:24,457 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.isValid(FlumeConfiguration.java:353)]
Post validation configuration for agent1
AgentConfiguration created without Configuration stubs for which only basic
syntactical validation was performed[agent1]
SOURCES: {r1={ parameters:{port=9001, handler.nickname=json handler,
channels=ch1, type=org.apache.flume.source.http.HTTPSource,
handler=org.apache.flume.source.http.JSONHandler} }
}
CHANNELS: {ch1={ parameters:{transactionCapacity=1000, capacity=10000,
type=memory} }}
SINKS: {k1={ parameters:{type=file_roll, channel=ch1,
sink.directory=C:\Projects\nickyk\apache-flume-1.3.1-bin\log} }}

2013-06-21 11:01:24,470 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:117)]
Channels:ch1

2013-06-21 11:01:24,471 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:118)]
Sinks k1

2013-06-21 11:01:24,471 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:119)]
Sources null

best regards,
nickolay kolev
On Fri, Jun 21, 2013 at 10:40 AM, Ashish <[EMAIL PROTECTED]> wrote:

> I am using Flume 1.3.1 and ran the example before sending.
>
> I don't have access to a windows system at the moment so can't test.
>
> This is output at my end on Flume Agent console
>
> 2013-06-21 13:03:54,913 (conf-file-poller-0) [INFO -
> org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.startAllComponents(DefaultLogicalNodeManager.java:152)]
> Starting Source src1
> 2013-06-21 13:03:54,927 (lifecycleSupervisor-1-2) [INFO -
> org.mortbay.log.Slf4jLog.info(Slf4jLog.java:67)] Logging to
> org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via
> org.mortbay.log.Slf4jLog
> 2013-06-21 13:03:54,958 (lifecycleSupervisor-1-2) [INFO -
> org.mortbay.log.Slf4jLog.info(Slf4jLog.java:67)] jetty-6.1.26
> 2013-06-21 13:03:55,022 (lifecycleSupervisor-1-2) [INFO -
> org.mortbay.log.Slf4jLog.info(Slf4jLog.java:67)] Started
> SocketConnector@0.0.0.0:9001
> 2013-06-21 13:09:36,992 (SinkRunner-PollingRunner-DefaultSinkProcessor)
> [INFO - org.apache.flume.sink.LoggerSink.process(LoggerSink.java:70)]
> Event: { headers:{timestamp=434324343, host=random_host.example.co
> m} body: 72 61 6E 64 6F 6D 5F 62 6F 64 79                random_body }
> 2013-06-21 13:09:36,992 (SinkRunner-PollingRunner-DefaultSinkProcessor)
> [INFO - org.apache.flume.sink.LoggerSink.process(LoggerSink.java:70)]
+
Ashish 2013-06-21, 08:25
+
Nickolay Kolev 2013-06-21, 07:52
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB