Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> Problems with Flume on windows with HTTP source


+
Nickolay Kolev 2013-06-20, 15:57
+
Hari Shreedharan 2013-06-20, 18:07
+
Nickolay Kolev 2013-06-21, 06:34
+
Ashish 2013-06-21, 06:48
+
Nickolay Kolev 2013-06-21, 07:03
+
Ashish 2013-06-21, 07:40
+
Nickolay Kolev 2013-06-21, 08:29
+
Nickolay Kolev 2013-06-21, 08:20
+
Ashish 2013-06-21, 08:25
Copy link to this message
-
Re: Problems with Flume on windows with HTTP source
Hi Ashish,

Thanks for the support. I will continue with this issue. Hope I will find
solution or at least confirmation that this is windows specific problem.
Have a nice day

best regards,
nickolay kolev
On Fri, Jun 21, 2013 at 10:40 AM, Ashish <[EMAIL PROTECTED]> wrote:

> I am using Flume 1.3.1 and ran the example before sending.
>
> I don't have access to a windows system at the moment so can't test.
>
> This is output at my end on Flume Agent console
>
> 2013-06-21 13:03:54,913 (conf-file-poller-0) [INFO -
> org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.startAllComponents(DefaultLogicalNodeManager.java:152)]
> Starting Source src1
> 2013-06-21 13:03:54,927 (lifecycleSupervisor-1-2) [INFO -
> org.mortbay.log.Slf4jLog.info(Slf4jLog.java:67)] Logging to
> org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via
> org.mortbay.log.Slf4jLog
> 2013-06-21 13:03:54,958 (lifecycleSupervisor-1-2) [INFO -
> org.mortbay.log.Slf4jLog.info(Slf4jLog.java:67)] jetty-6.1.26
> 2013-06-21 13:03:55,022 (lifecycleSupervisor-1-2) [INFO -
> org.mortbay.log.Slf4jLog.info(Slf4jLog.java:67)] Started
> SocketConnector@0.0.0.0:9001
> 2013-06-21 13:09:36,992 (SinkRunner-PollingRunner-DefaultSinkProcessor)
> [INFO - org.apache.flume.sink.LoggerSink.process(LoggerSink.java:70)]
> Event: { headers:{timestamp=434324343, host=random_host.example.co
> m} body: 72 61 6E 64 6F 6D 5F 62 6F 64 79                random_body }
> 2013-06-21 13:09:36,992 (SinkRunner-PollingRunner-DefaultSinkProcessor)
> [INFO - org.apache.flume.sink.LoggerSink.process(LoggerSink.java:70)]
> Event: { headers:{namenode=namenode.example.com, datanode> random_datanode.example.com} body: 72 65 61 6C 6C 79 5F 72 61 6E 64 6F 6D
> 5F 62 6F really_random_bo }
>
> Flume dev's can comment further on this.
>
>
> On Fri, Jun 21, 2013 at 12:33 PM, Nickolay Kolev <[EMAIL PROTECTED]>wrote:
>
>> Hi Ashish,
>>
>> Thanks for pointing me that error. I am trying to read the code and this
>> is the correct full class name. (last time I wrote java code was in 1998
>> and my knowledge are a lot out of date)
>>
>> Unfortunately the result is the same.
>> >curl -X POST -H "Content-Type: application/json" -d "[{ \"headers\" : {
>> \"timestamp\" : \"434324343\", \"host\" : \"random_host.example.com\" },
>> \"body\" : \"random_body\" }, { \"headers\" : { \"namenode\" : \"
>> namenode.example.com\", \"datanode\" : \"random_datanode.example.com\"
>> }, \"body\" : \"really_random_body\"}]" http://localhost:9001
>>
>> curl: (7) Failed connect to localhost:9001; No error
>>
>> Could be the reason if this errors the fact that I use Windows? I see in
>> wiki that Windows is not officially supported.
>>
>> Yesterday my first test was with netcat source and it worked as expected.
>>
>>
>> best regards,
>> nickolay kolev
>>
>>
>> On Fri, Jun 21, 2013 at 9:48 AM, Ashish <[EMAIL PROTECTED]> wrote:
>>
>>> update the handler config as
>>>
>>> agent1.sources.r1.handler = org.apache.flume.source.http.JSONHandler
>>>
>>> and try. I used your source config and it worked for me. Let us know if
>>> it work
>>>
>>>
>>> On Fri, Jun 21, 2013 at 12:04 PM, Nickolay Kolev <[EMAIL PROTECTED]>wrote:
>>>
>>>> Hi Hari and the all members of this list,
>>>>
>>>> Thanks for pointing me that error. This is actually my second attempt
>>>> to use Apache Flume - with avro client. The symptoms are the same - the is
>>>> no listener on the specified port and I am not able to connect.
>>>>
>>>> I started to evaluate Flume just yesterday and many things are new to
>>>> me and probably I do many mistakes. I want to point that I use Flume 1.3.1
>>>> compiled by me if this is important to be mentioned. I have tested too with
>>>> the official binary from Flume web site but the results are the same.
>>>>
>>>> Here is what I do to test HTTP source:
>>>>
>>>> my config file (httppost.conf):
>>>> agent1.sources = r1
>>>> agent1.channels = ch1
>>>> agent1.sinks = k1
>>>>
>>>> agent1.sources.r1.type = org.apache.flume.source.http.HTTPSource
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB