Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume, mail # user - Problems with Flume on windows with HTTP source


+
Nickolay Kolev 2013-06-20, 15:57
+
Hari Shreedharan 2013-06-20, 18:07
+
Nickolay Kolev 2013-06-21, 06:34
+
Ashish 2013-06-21, 06:48
+
Nickolay Kolev 2013-06-21, 07:03
+
Ashish 2013-06-21, 07:40
+
Nickolay Kolev 2013-06-21, 08:29
Copy link to this message
-
Re: Problems with Flume on windows with HTTP source
Nickolay Kolev 2013-06-21, 08:20
Hi Asish,

I think I found the problem - there must be something wrong with the
configuration and the source is not created. Unfortunately I can not figure
what.
Did you test it with my configuration?

Here is what I found in server log:
2013-06-21 11:01:24,433 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.isValid(FlumeConfiguration.java:295)]
Starting validation of configuration for agent: agent1,
initial-configuration: AgentConfiguration[agent1]
SOURCES: {r1={ parameters:{port=9001, handler.nickname=json handler,
channels=ch1, type=org.apache.flume.source.http.HTTPSource,
handler=org.apache.flume.source.http.JSONHandler} }
}
CHANNELS: {ch1={ parameters:{transactionCapacity=1000, capacity=10000,
type=memory} }}
SINKS: {k1={ parameters:{type=file_roll, channel=ch1,
sink.directory=C:\Projects\nickyk\apache-flume-1.3.1-bin\log} }}

2013-06-21 11:01:24,451 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.validateChannels(FlumeConfiguration.java:450)]
Created channel ch1

2013-06-21 11:01:24,451 (conf-file-poller-0) [WARN -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.validateSources(FlumeConfiguration.java:489)]
Agent configuration for 'agent1' has no sources.
2013-06-21 11:01:24,455 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.validateSinks(FlumeConfiguration.java:655)]
Creating sink: k1 using FILE_ROLL
2013-06-21 11:01:24,457 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.isValid(FlumeConfiguration.java:353)]
Post validation configuration for agent1
AgentConfiguration created without Configuration stubs for which only basic
syntactical validation was performed[agent1]
SOURCES: {r1={ parameters:{port=9001, handler.nickname=json handler,
channels=ch1, type=org.apache.flume.source.http.HTTPSource,
handler=org.apache.flume.source.http.JSONHandler} }
}
CHANNELS: {ch1={ parameters:{transactionCapacity=1000, capacity=10000,
type=memory} }}
SINKS: {k1={ parameters:{type=file_roll, channel=ch1,
sink.directory=C:\Projects\nickyk\apache-flume-1.3.1-bin\log} }}

2013-06-21 11:01:24,470 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:117)]
Channels:ch1

2013-06-21 11:01:24,471 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:118)]
Sinks k1

2013-06-21 11:01:24,471 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:119)]
Sources null

best regards,
nickolay kolev
On Fri, Jun 21, 2013 at 10:40 AM, Ashish <[EMAIL PROTECTED]> wrote:

> I am using Flume 1.3.1 and ran the example before sending.
>
> I don't have access to a windows system at the moment so can't test.
>
> This is output at my end on Flume Agent console
>
> 2013-06-21 13:03:54,913 (conf-file-poller-0) [INFO -
> org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.startAllComponents(DefaultLogicalNodeManager.java:152)]
> Starting Source src1
> 2013-06-21 13:03:54,927 (lifecycleSupervisor-1-2) [INFO -
> org.mortbay.log.Slf4jLog.info(Slf4jLog.java:67)] Logging to
> org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via
> org.mortbay.log.Slf4jLog
> 2013-06-21 13:03:54,958 (lifecycleSupervisor-1-2) [INFO -
> org.mortbay.log.Slf4jLog.info(Slf4jLog.java:67)] jetty-6.1.26
> 2013-06-21 13:03:55,022 (lifecycleSupervisor-1-2) [INFO -
> org.mortbay.log.Slf4jLog.info(Slf4jLog.java:67)] Started
> SocketConnector@0.0.0.0:9001
> 2013-06-21 13:09:36,992 (SinkRunner-PollingRunner-DefaultSinkProcessor)
> [INFO - org.apache.flume.sink.LoggerSink.process(LoggerSink.java:70)]
> Event: { headers:{timestamp=434324343, host=random_host.example.co
> m} body: 72 61 6E 64 6F 6D 5F 62 6F 64 79                random_body }
> 2013-06-21 13:09:36,992 (SinkRunner-PollingRunner-DefaultSinkProcessor)
> [INFO - org.apache.flume.sink.LoggerSink.process(LoggerSink.java:70)]
+
Ashish 2013-06-21, 08:25
+
Nickolay Kolev 2013-06-21, 07:52