Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume, mail # user - How to use HTTPS with HTTPSource?


Copy link to this message
-
Re: How to use HTTPS with HTTPSource?
Ashish 2013-06-24, 15:05
So am I :) I am just into my 2nd week of reading about Flume. If I were in
your place, I would have added Jetty SSL example code to HTTP source and
make it work. Other approaches are purist approaches, and are needed only
when you need a very low level control of your implementation.

I can try to put up an example by this weekend, if time permits.

HTH !

On Mon, Jun 24, 2013 at 8:24 PM, shushuai zhu <[EMAIL PROTECTED]> wrote:

> Ashish, thanks again. Could you elaborate a little more what I should do?
> I am relatively new to Flume (just started using it a couple of weeks ago)
> and also new to open source code base.
>
> Shushuai
>    *From:* Ashish <[EMAIL PROTECTED]>
> *To:* [EMAIL PROTECTED]; shushuai zhu <[EMAIL PROTECTED]>
> *Sent:* Friday, June 21, 2013 10:38 PM
>
> *Subject:* Re: How to use HTTPS with HTTPSource?
>
> HTTP source has not changed much :) One way to patch in additional SSL
> stuff in jetty code. The second more tedious way is to use NIO framework
> with SSL code like Netty or MINA. We are working on MINA M1 release and
> perhaps after that shall take a tab at writing an HTTP source based on that.
>
> HTH !
>
>
> On Sat, Jun 22, 2013 at 3:07 AM, shushuai zhu <[EMAIL PROTECTED]> wrote:
>
> Ashish,
>
> Thanks for the pointer. So I will create my own source, say HTTPSSource,
> which extends the HTPSource to add the https connection, then use the
> custom HTTPSSource in flume.conf.
>
> Is this the right source code you referred (I am using flume 1.3.1)?
>
>
> https://git-wip-us.apache.org/repos/asf?p=flume.git;a=tree;f=flume-ng-core/src/main/java/org/apache/flume/source/http;h=fa5477033ca23c075fec6af2c391b01d1d49f69a;hb=trunk
>
>
> Shushuai
>
>    *From:* Ashish <[EMAIL PROTECTED]>
> *To:* [EMAIL PROTECTED]; shushuai zhu <[EMAIL PROTECTED]>
> *Sent:* Friday, June 21, 2013 5:09 AM
> *Subject:* Re: How to use HTTPS with HTTPSource?
>
> AFAIK, HTTPSource doesn't support HTTPS. HTTP Source uses embedded jetty,
> so it would extending HTTP source to add SSL configuration for this.
>
> This link should help
> http://git.eclipse.org/c/jetty/org.eclipse.jetty.project.git/tree/examples/embedded/src/main/java/org/eclipse/jetty/embedded/ManyConnectors.java
>
> It should be just few lines of code change.
>
> Please note I am referring to source code in master branch.
>
>
> On Thu, Jun 20, 2013 at 1:40 AM, shushuai zhu <[EMAIL PROTECTED]> wrote:
>
> Hi,
>
> I am using HTTP to transfer data from remote machine to HTTPSource of
> Flume agent (the flume conf. file looks like below). Is HTTPS supported?
> Anyone has examples to use HTTPS to transfer the data?
>
> agent1.sources = r1
> agent1.sources.r1.type = org.apache.flume.source.http.HTTPSource
> agent1.sources.r1.port = 41415
> agent1.sources.r1.channels = ch1
> agent1.sources.r1.handler = org.apache.flume.source.http.JSONHandler
> Thanks.
>
> Shushuai
>
>
>
>
>
> --
> thanks
> ashish
>
> Blog: http://www.ashishpaliwal.com/blog
> My Photo Galleries: http://www.pbase.com/ashishpaliwal
>
>
>
>
>
> --
> thanks
> ashish
>
> Blog: http://www.ashishpaliwal.com/blog
> My Photo Galleries: http://www.pbase.com/ashishpaliwal
>
>
>
--
thanks
ashish

Blog: http://www.ashishpaliwal.com/blog
My Photo Galleries: http://www.pbase.com/ashishpaliwal