Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Problems with Flume on windows with HTTP source


Copy link to this message
-
Re: Problems with Flume on windows with HTTP source
I pulled your source configuration. Here is the full configuration
(single-node-custom-source.properties), removed my custom source from here

# Exampple for single node config

# Base Config
a1.sources=src1
a1.sinks=sink1
a1.channels=ch1

# Configure the source
a1.sources.src1.type = org.apache.flume.source.http.HTTPSource
a1.sources.src1.port = 9001
a1.sources.src1.bind = localhost
a1.sources.src1.handler = org.apache.flume.source.http.JSONHandler

# Sink Configuration
a1.sinks.sink1.type=logger

# Channel configuration
a1.channels.ch1.type=memory
a1.channels.ch1.capacity=1000
a1.channels.ch1.transactionCapacity=100

# Link stuff together
a1.sources.src1.channels=ch1
a1.sinks.sink1.channel=ch1

and this is how I start my agent

bin/flume-ng agent --conf-file
conf/single-node-custom-source.properties --name a1 --conf ./conf/
-Dflume.root.logger=INFO,console
On Fri, Jun 21, 2013 at 1:50 PM, Nickolay Kolev <[EMAIL PROTECTED]> wrote:

> Hi Asish,
>
> I think I found the problem - there must be something wrong with the
> configuration and the source is not created. Unfortunately I can not figure
> what.
> Did you test it with my configuration?
>
> Here is what I found in server log:
> 2013-06-21 11:01:24,433 (conf-file-poller-0) [DEBUG -
> org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.isValid(FlumeConfiguration.java:295)]
> Starting validation of configuration for agent: agent1,
> initial-configuration: AgentConfiguration[agent1]
> SOURCES: {r1={ parameters:{port=9001, handler.nickname=json handler,
> channels=ch1, type=org.apache.flume.source.http.HTTPSource,
> handler=org.apache.flume.source.http.JSONHandler} }
> }
> CHANNELS: {ch1={ parameters:{transactionCapacity=1000, capacity=10000,
> type=memory} }}
> SINKS: {k1={ parameters:{type=file_roll, channel=ch1,
> sink.directory=C:\Projects\nickyk\apache-flume-1.3.1-bin\log} }}
>
> 2013-06-21 11:01:24,451 (conf-file-poller-0) [DEBUG -
> org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.validateChannels(FlumeConfiguration.java:450)]
> Created channel ch1
>
> 2013-06-21 11:01:24,451 (conf-file-poller-0) [WARN -
> org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.validateSources(FlumeConfiguration.java:489)]
> Agent configuration for 'agent1' has no sources.
> 2013-06-21 11:01:24,455 (conf-file-poller-0) [DEBUG -
> org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.validateSinks(FlumeConfiguration.java:655)]
> Creating sink: k1 using FILE_ROLL
> 2013-06-21 11:01:24,457 (conf-file-poller-0) [DEBUG -
> org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.isValid(FlumeConfiguration.java:353)]
> Post validation configuration for agent1
> AgentConfiguration created without Configuration stubs for which only
> basic syntactical validation was performed[agent1]
> SOURCES: {r1={ parameters:{port=9001, handler.nickname=json handler,
> channels=ch1, type=org.apache.flume.source.http.HTTPSource,
> handler=org.apache.flume.source.http.JSONHandler} }
> }
> CHANNELS: {ch1={ parameters:{transactionCapacity=1000, capacity=10000,
> type=memory} }}
> SINKS: {k1={ parameters:{type=file_roll, channel=ch1,
> sink.directory=C:\Projects\nickyk\apache-flume-1.3.1-bin\log} }}
>
> 2013-06-21 11:01:24,470 (conf-file-poller-0) [DEBUG -
> org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:117)]
> Channels:ch1
>
> 2013-06-21 11:01:24,471 (conf-file-poller-0) [DEBUG -
> org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:118)]
> Sinks k1
>
> 2013-06-21 11:01:24,471 (conf-file-poller-0) [DEBUG -
> org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:119)]
> Sources null
>
> best regards,
> nickolay kolev
>
>
>
>
> On Fri, Jun 21, 2013 at 10:40 AM, Ashish <[EMAIL PROTECTED]> wrote:
>
>> I am using Flume 1.3.1 and ran the example before sending.
>>
>> I don't have access to a windows system at the moment so can't test.
>>
>> This is output at my end on Flume Agent console
thanks
ashish

Blog: http://www.ashishpaliwal.com/blog
My Photo Galleries: http://www.pbase.com/ashishpaliwal
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB