Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # dev >> Re: [jira] [Commented] (FLUME-1424) File Channel should support encryption


+
Mike Percy 2012-08-10, 22:22
+
Ralph Goers 2012-08-10, 22:31
+
Arvind Prabhakar 2012-08-10, 23:07
+
Mike Percy 2012-08-12, 23:00
+
Ralph Goers 2012-08-20, 18:55
Copy link to this message
-
Re: [jira] [Commented] (FLUME-1424) File Channel should support encryption
Hi Ralph,
I have been looking into this, although a couple other things popped up
last week for me also.

I think we should wait for
https://issues.apache.org/jira/browse/FLUME-1487 (Brock
has a patch up for review) for this issue for the following reasons:
1. The metadata exposed in FLUME-1487 will allow us to more easily create
multiple implementations that take care of different encryption use cases /
requirements.
2. Without FLUME-1487, all of the information about the encryption scheme
used will have to be embedded in configuration and will not be possible to
validate from the encryption plugin when reading the file.

Regards,
Mike

On Mon, Aug 20, 2012 at 11:55 AM, Ralph Goers <[EMAIL PROTECTED]>wrote:

> Mike,
>
> Have you made any progress on this? Our engineer didn't take this on
> because he saw that you had volunteered. If you are working on something
> else he needs to jump on this right away.
>
> Ralph
>
> On Aug 12, 2012, at 4:00 PM, Mike Percy wrote:
>
> > Thanks Ralph and Arvind for the pointers. I'll take a look and see if I
> can
> > come up with something simple but extensible.
> >
> > Regards,
> > Mike
> >
> > On Fri, Aug 10, 2012 at 4:07 PM, Arvind Prabhakar <[EMAIL PROTECTED]>
> wrote:
> >
> >> @Mike - please go ahead. I have done some preliminary analysis but no
> patch
> >> yet. It is similar to what Ralph has suggested below but not based on
> >> input/output streams. My findings are that it is possible to use built
> in
> >> JCE supported algorithms such as PBEWithMD5AndDES but have not yet done
> a
> >> performance impact analysis.
> >>
> >> Regards,
> >> Arvind Prabhakar
> >>
> >> On Fri, Aug 10, 2012 at 3:31 PM, Ralph Goers <
> [EMAIL PROTECTED]
> >>> wrote:
> >>
> >>> It would be great for you to pick this up.  We need this at Intuit asap
> >>> and we are planning on having one of our engineers work on it next week
> >> if
> >>> no one here can do it first.  What I had suggested  was to have add a
> >>> FlumeEventFactory that could be configurable.  We could then use an
> >>> EncryptedFlumeEventFactory that wraps the OutputStream with a
> >>> CipherOutputStream and likewise when reading.  Ideally, the key should
> be
> >>> able to be stored either in the default keystore something pluggable.
> >>>
> >>> Ralph
> >>>
> >>> On Aug 10, 2012, at 3:22 PM, Mike Percy wrote:
> >>>
> >>>> @Arvind: JIRA is down but I would like to look @ this issue. Are you
> >>>> working on a patch or can I pick this up?
> >>>>
> >>>> Regards,
> >>>> Mike
> >>>>
> >>>> On Tue, Aug 7, 2012 at 12:37 AM, Arvind Prabhakar (JIRA) <
> >>> [EMAIL PROTECTED]>wrote:
> >>>>
> >>>>>
> >>>>>   [
> >>>>>
> >>>
> >>
> https://issues.apache.org/jira/browse/FLUME-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13429992#comment-13429992
> >>> ]
> >>>>>
> >>>>> Arvind Prabhakar commented on FLUME-1424:
> >>>>> -----------------------------------------
> >>>>>
> >>>>> Yes, the put records do store the data in them. We can perhaps start
> >>> with
> >>>>> that as a first step and if more requirements pop-up, we can address
> >>> them
> >>>>> in follow-up Jiras as necessary.
> >>>>>
> >>>>>> File Channel should support encryption
> >>>>>> --------------------------------------
> >>>>>>
> >>>>>>               Key: FLUME-1424
> >>>>>>               URL: https://issues.apache.org/jira/browse/FLUME-1424
> >>>>>>           Project: Flume
> >>>>>>        Issue Type: Bug
> >>>>>>          Reporter: Arvind Prabhakar
> >>>>>>          Assignee: Arvind Prabhakar
> >>>>>>
> >>>>>> When persisting the data to disk, the File Channel should allow some
> >>>>> form of encryption to ensure safety of data.
> >>>>>
> >>>>> --
> >>>>> This message is automatically generated by JIRA.
> >>>>> If you think it was sent incorrectly, please contact your JIRA
> >>>>> administrators:
> >>>>>
> >>>
> https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
> >>>>> For more information on JIRA, see: