Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Problems with Flume on windows with HTTP source


Copy link to this message
-
Re: Problems with Flume on windows with HTTP source
update the handler config as

agent1.sources.r1.handler = org.apache.flume.source.http.JSONHandler

and try. I used your source config and it worked for me. Let us know if it
work
On Fri, Jun 21, 2013 at 12:04 PM, Nickolay Kolev <[EMAIL PROTECTED]> wrote:

> Hi Hari and the all members of this list,
>
> Thanks for pointing me that error. This is actually my second attempt to
> use Apache Flume - with avro client. The symptoms are the same - the is no
> listener on the specified port and I am not able to connect.
>
> I started to evaluate Flume just yesterday and many things are new to me
> and probably I do many mistakes. I want to point that I use Flume 1.3.1
> compiled by me if this is important to be mentioned. I have tested too with
> the official binary from Flume web site but the results are the same.
>
> Here is what I do to test HTTP source:
>
> my config file (httppost.conf):
> agent1.sources = r1
> agent1.channels = ch1
> agent1.sinks = k1
>
> agent1.sources.r1.type = org.apache.flume.source.http.HTTPSource
> agent1.sources.r1.port = 9001
> agent1.sources.r1.channels = ch1
> #agent1.sources.r1.handler = org.example.rest.RestHandler
> agent1.sources.r1.handler = org.apache.flume.http.JSONHandler
> agent1.sources.r1.handler.nickname = json handler
>
> agent1.sinks.k1.type = file_roll
> agent1.sinks.k1.channel = ch1
>
> agent1.sinks.k1.sink.directory > C:\\Projects\\nickyk\\apache-flume-1.3.1-bin\\log
>
> agent1.channels.ch1.type = memory
> agent1.channels.ch1.capacity = 10000
> agent1.channels.ch1.transactionCapacity = 1000
>
> my server bat file:
>
> set FLUME_HOME=C:\Projects\nickyk\apache-flume-1.3.1-bin\
> set JAVA="C:\Java\jdk1.6.0_45\bin\java.exe"
> set JAVA_OPTS=-Xmx60m
> set CONF=%FLUME_HOME%\conf\httppost.conf
> set AGENT=agent1
>
> %JAVA% %JAVA_OPTS%
> -Dlog4j.configuration=file:///%FLUME_HOME%\conf\log4j.properties -cp
> "%FLUME_HOME%\lib\*" org.apache.flume.node.Application -f %CONF% -n %AGENT%
>
> How I test:
>
> server console:
> >"C:\Java\jdk1.6.0_45\bin\java.exe" -Xmx60m
> -Dlog4j.configuration=file:///C:\Projects\nickyk\apache-flume-1.3.1-bin\\conf\log4j.properties
> -cp "C:\Projects\nickyk\apache-flume-1.3.1-bin\\lib\*"
> org.apache.flume.node.Application -f
> C:\Projects\nickyk\apache-flume-1.3.1-bin\\conf\httppost.conf -n agent1
>
> client console:
> >C:\Projects\nickyk\apache-flume-1.3.1-bin>curl -X POST -H "Content-Type:
> application/json" -d "[{ \"headers\" : { \"timestamp\" : \"434324343\",
> \"host\" : \"random_host.example.co
> m\" }, \"body\" : \"random_body\" }, { \"headers\" : { \"namenode\" : \"
> namenode.example.com\", \"datanode\" : \"random_datanode.example.com\" },
> \"body\" : \"really_random_body\"
> }]" http://localhost:9001
> curl: (7) Failed connect to localhost:9001; No error
>
> If I understand correct the documentation I should expect that on the
> specified port (9001) it must be listener that will accept POST requests?
> Is this correct?
> Do I miss some important aspects that are critical to run Flume?
>
> Thanks in advance for your help.
>
>
> best regards,
> nickolay kolev
>
>
>
> On Thu, Jun 20, 2013 at 9:07 PM, Hari Shreedharan <
> [EMAIL PROTECTED]> wrote:
>
>>  Does not look like you are using the http source at all. Your source
>> type needs to be HTTP
>>
>>
>> Cheers,
>> Hari
>>
>> On Thursday, June 20, 2013 at 8:57 AM, Nickolay Kolev wrote:
>>
>> Hi all,
>> I am new to flume and all that logging stuff and probably many things are
>> unclear to me despite I read the docs.
>> I want to run Flume on windows server to create proof of concept
>> solution. I was able to compile it successfully. I was able to make
>> successful test with netcat.
>> Next test that i wanted to do is with http source but unfortunately it
>> did not pass.
>>  Here is what I do:
>>
>> my server config (httppost.conf):
>> agent1.sources = r1
>> agent1.channels = memoryChannel
>> agent1.sinks = k1
>>
>> agent1.sources.r1.type = avro
>> agent1.sources.r1.bind = localhost
>> agent1.sources.r1.port = 9003
thanks
ashish

Blog: http://www.ashishpaliwal.com/blog
My Photo Galleries: http://www.pbase.com/ashishpaliwal