Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume, mail # user - How to use HTTPS with HTTPSource?


+
shushuai zhu 2013-06-19, 20:10
+
Josh Myers 2013-06-19, 20:16
+
Ashish 2013-06-21, 09:09
+
shushuai zhu 2013-06-21, 21:37
+
Ashish 2013-06-22, 02:38
+
shushuai zhu 2013-06-24, 14:54
+
Ashish 2013-06-24, 15:05
+
Hari Shreedharan 2013-06-24, 23:04
+
Ashish 2013-06-26, 09:28
+
shushuai zhu 2013-06-26, 14:43
+
Ashish 2013-06-27, 01:50
Copy link to this message
-
Re: How to use HTTPS with HTTPSource?
Ashish 2013-06-28, 11:00
Have added the patch to JIRA. Let's wait for the review.
On Thu, Jun 27, 2013 at 7:20 AM, Ashish <[EMAIL PROTECTED]> wrote:

> Sure, I am working on the test cases and Httpclient is giving me tough
> time with SSL. If it doesn't work, shall write simple SSL client to test it.
>
>
> On Wed, Jun 26, 2013 at 8:13 PM, shushuai zhu <[EMAIL PROTECTED]> wrote:
>
>> Ashish, thx. Will try your solution. Please also kindly send a
>> notice after you submit the patch.
>>
>> Shushuai
>>    *From:* Ashish <[EMAIL PROTECTED]>
>> *To:* [EMAIL PROTECTED]; shushuai zhu <[EMAIL PROTECTED]>
>> *Sent:* Wednesday, June 26, 2013 5:28 AM
>>
>> *Subject:* Re: How to use HTTPS with HTTPSource?
>>
>> The example I shared works for Jetty 9, Flume uses 6.1.26.
>>
>> Here is a basic version. Add these lines in HTTPSource in start() method.
>> This is a basic version. You would like to read these properties from flume
>> config in configure(). This is small stuff.
>>
>> SslSocketConnector sslSocketConnector = new SslSocketConnector();
>> sslSocketConnector.setKeystore("jettykeystore");
>> sslSocketConnector.setKeyPassword("password");
>> sslSocketConnector.setPort(8443);
>> srv.setConnectors(new Connector[] { connector, sslSocketConnector });
>>
>> Here is how it would like
>>
>>   @Override
>>   public void start() {
>>     Preconditions.checkState(srv == null,
>>             "Running HTTP Server found in source: " + getName()
>>             + " before I started one."
>>             + "Will not attempt to start.");
>>     srv = new Server();
>>     SocketConnector connector = new SocketConnector();
>>     connector.setPort(port);
>>     connector.setHost(host);
>>
>>         SslSocketConnector sslSocketConnector = new SslSocketConnector();
>>         sslSocketConnector.setKeystore("jettykeystore");
>>         sslSocketConnector.setKeyPassword("password");
>>         sslSocketConnector.setPort(8443);
>>
>>     srv.setConnectors(new Connector[] { connector, sslSocketConnector });
>>     try {
>>       org.mortbay.jetty.servlet.Context root >>               new org.mortbay.jetty.servlet.Context(
>>               srv, "/", org.mortbay.jetty.servlet.Context.SESSIONS);
>>       root.addServlet(new ServletHolder(new FlumeHTTPServlet()), "/");
>>       srv.start();
>> ........
>>
>> To submit a patch, would need to refine the code a bit and add test
>> cases. Shall take a while. HTH !
>>
>>
>> On Mon, Jun 24, 2013 at 8:24 PM, shushuai zhu <[EMAIL PROTECTED]> wrote:
>>
>> Ashish, thanks again. Could you elaborate a little more what I should do?
>> I am relatively new to Flume (just started using it a couple of weeks ago)
>> and also new to open source code base.
>>
>> Shushuai
>>    *From:* Ashish <[EMAIL PROTECTED]>
>> *To:* [EMAIL PROTECTED]; shushuai zhu <[EMAIL PROTECTED]>
>> *Sent:* Friday, June 21, 2013 10:38 PM
>>
>> *Subject:* Re: How to use HTTPS with HTTPSource?
>>
>> HTTP source has not changed much :) One way to patch in additional SSL
>> stuff in jetty code. The second more tedious way is to use NIO framework
>> with SSL code like Netty or MINA. We are working on MINA M1 release and
>> perhaps after that shall take a tab at writing an HTTP source based on that.
>>
>> HTH !
>>
>>
>> On Sat, Jun 22, 2013 at 3:07 AM, shushuai zhu <[EMAIL PROTECTED]> wrote:
>>
>> Ashish,
>>
>> Thanks for the pointer. So I will create my own source, say HTTPSSource,
>> which extends the HTPSource to add the https connection, then use the
>> custom HTTPSSource in flume.conf.
>>
>> Is this the right source code you referred (I am using flume 1.3.1)?
>>
>>
>> https://git-wip-us.apache.org/repos/asf?p=flume.git;a=tree;f=flume-ng-core/src/main/java/org/apache/flume/source/http;h=fa5477033ca23c075fec6af2c391b01d1d49f69a;hb=trunk
>>
>>
>> Shushuai
>>
>>    *From:* Ashish <[EMAIL PROTECTED]>
>> *To:* [EMAIL PROTECTED]; shushuai zhu <[EMAIL PROTECTED]>
>> *Sent:* Friday, June 21, 2013 5:09 AM
>> *Subject:* Re: How to use HTTPS with HTTPSource?
>>
>> AFAIK, HTTPSource doesn't support HTTPS. HTTP Source uses embedded jetty,

thanks
ashish

Blog: http://www.ashishpaliwal.com/blog
My Photo Galleries: http://www.pbase.com/ashishpaliwal
+
Israel Ekpo 2013-06-24, 17:09