Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Problems with Flume on windows with HTTP source


Copy link to this message
-
Re: Problems with Flume on windows with HTTP source
I pulled your source configuration. Here is the full configuration
(single-node-custom-source.properties), removed my custom source from here

# Exampple for single node config

# Base Config
a1.sources=src1
a1.sinks=sink1
a1.channels=ch1

# Configure the source
a1.sources.src1.type = org.apache.flume.source.http.HTTPSource
a1.sources.src1.port = 9001
a1.sources.src1.bind = localhost
a1.sources.src1.handler = org.apache.flume.source.http.JSONHandler

# Sink Configuration
a1.sinks.sink1.type=logger

# Channel configuration
a1.channels.ch1.type=memory
a1.channels.ch1.capacity=1000
a1.channels.ch1.transactionCapacity=100

# Link stuff together
a1.sources.src1.channels=ch1
a1.sinks.sink1.channel=ch1

and this is how I start my agent

bin/flume-ng agent --conf-file
conf/single-node-custom-source.properties --name a1 --conf ./conf/
-Dflume.root.logger=INFO,console
On Fri, Jun 21, 2013 at 1:50 PM, Nickolay Kolev <[EMAIL PROTECTED]> wrote:

> Hi Asish,
>
> I think I found the problem - there must be something wrong with the
> configuration and the source is not created. Unfortunately I can not figure
> what.
> Did you test it with my configuration?
>
> Here is what I found in server log:
> 2013-06-21 11:01:24,433 (conf-file-poller-0) [DEBUG -
> org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.isValid(FlumeConfiguration.java:295)]
> Starting validation of configuration for agent: agent1,
> initial-configuration: AgentConfiguration[agent1]
> SOURCES: {r1={ parameters:{port=9001, handler.nickname=json handler,
> channels=ch1, type=org.apache.flume.source.http.HTTPSource,
> handler=org.apache.flume.source.http.JSONHandler} }
> }
> CHANNELS: {ch1={ parameters:{transactionCapacity=1000, capacity=10000,
> type=memory} }}
> SINKS: {k1={ parameters:{type=file_roll, channel=ch1,
> sink.directory=C:\Projects\nickyk\apache-flume-1.3.1-bin\log} }}
>
> 2013-06-21 11:01:24,451 (conf-file-poller-0) [DEBUG -
> org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.validateChannels(FlumeConfiguration.java:450)]
> Created channel ch1
>
> 2013-06-21 11:01:24,451 (conf-file-poller-0) [WARN -
> org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.validateSources(FlumeConfiguration.java:489)]
> Agent configuration for 'agent1' has no sources.
> 2013-06-21 11:01:24,455 (conf-file-poller-0) [DEBUG -
> org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.validateSinks(FlumeConfiguration.java:655)]
> Creating sink: k1 using FILE_ROLL
> 2013-06-21 11:01:24,457 (conf-file-poller-0) [DEBUG -
> org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.isValid(FlumeConfiguration.java:353)]
> Post validation configuration for agent1
> AgentConfiguration created without Configuration stubs for which only
> basic syntactical validation was performed[agent1]
> SOURCES: {r1={ parameters:{port=9001, handler.nickname=json handler,
> channels=ch1, type=org.apache.flume.source.http.HTTPSource,
> handler=org.apache.flume.source.http.JSONHandler} }
> }
> CHANNELS: {ch1={ parameters:{transactionCapacity=1000, capacity=10000,
> type=memory} }}
> SINKS: {k1={ parameters:{type=file_roll, channel=ch1,
> sink.directory=C:\Projects\nickyk\apache-flume-1.3.1-bin\log} }}
>
> 2013-06-21 11:01:24,470 (conf-file-poller-0) [DEBUG -
> org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:117)]
> Channels:ch1
>
> 2013-06-21 11:01:24,471 (conf-file-poller-0) [DEBUG -
> org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:118)]
> Sinks k1
>
> 2013-06-21 11:01:24,471 (conf-file-poller-0) [DEBUG -
> org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:119)]
> Sources null
>
> best regards,
> nickolay kolev
>
>
>
>
> On Fri, Jun 21, 2013 at 10:40 AM, Ashish <[EMAIL PROTECTED]> wrote:
>
>> I am using Flume 1.3.1 and ran the example before sending.
>>
>> I don't have access to a windows system at the moment so can't test.
>>
>> This is output at my end on Flume Agent console
thanks
ashish

Blog: http://www.ashishpaliwal.com/blog
My Photo Galleries: http://www.pbase.com/ashishpaliwal