HBase, mail # dev - Security patches: HBASE-2742 and HBASE-3025

Re: Security patches: HBASE-2742 and HBASE-3025

Ted Yu 2011-11-17, 18:40

Gary:
I still have a little comment for 2742.
Pardon my slow typing. I should fix them before noon.

Thanks

On Thu, Nov 17, 2011 at 10:33 AM, Gary Helmling <[EMAIL PROTECTED]> wrote:

> Hi all,
>
> I intend to commit the following patches for HBase security to the
> 0.92 branch and trunk at the end of day today:
>
> HBASE-2742: Provide a secure RPC engine for HBase
> https://reviews.apache.org/r/1991/
>
> HBASE-3025: Coprocessor based simple access control
> https://reviews.apache.org/r/2041/
>
> If you have not reviewed and want to, or feel like you need more time,
> please take a look today or speak up now.
>
>
> --gh
>

Re: Security patches: HBASE-2742 and HBASE-3025

Stack 2011-11-17, 18:44

On Thu, Nov 17, 2011 at 10:33 AM, Gary Helmling <[EMAIL PROTECTED]> wrote:
> Hi all,
>
> I intend to commit the following patches for HBase security to the
> 0.92 branch and trunk at the end of day today:
>
> HBASE-2742: Provide a secure RPC engine for HBase
> https://reviews.apache.org/r/1991/
>
> HBASE-3025: Coprocessor based simple access control
> https://reviews.apache.org/r/2041/
>
> If you have not reviewed and want to, or feel like you need more time,
> please take a look today or speak up now.
>

Am reviewing now.

That would leave Andrew's secure zk issue to complete our security story?

St.Ack

Re: Security patches: HBASE-2742 and HBASE-3025

Gary Helmling 2011-11-17, 18:47

Yes, the ZK authentication in 2418 is the final piece of the puzzle.
On Thu, Nov 17, 2011 at 10:44 AM, Stack <[EMAIL PROTECTED]> wrote:
> On Thu, Nov 17, 2011 at 10:33 AM, Gary Helmling <[EMAIL PROTECTED]> wrote:
>> Hi all,
>>
>> I intend to commit the following patches for HBase security to the
>> 0.92 branch and trunk at the end of day today:
>>
>> HBASE-2742: Provide a secure RPC engine for HBase
>> https://reviews.apache.org/r/1991/
>>
>> HBASE-3025: Coprocessor based simple access control
>> https://reviews.apache.org/r/2041/
>>
>> If you have not reviewed and want to, or feel like you need more time,
>> please take a look today or speak up now.
>>
>
> Am reviewing now.
>
> That would leave Andrew's secure zk issue to complete our security story?
>
> St.Ack
>

Re: Security patches: HBASE-2742 and HBASE-3025

Ted Yu 2011-11-17, 18:58

I am still going over changes in 2742. But I don't expect to find much.

2742 and 3025 haven't gone through HadoopQA yet.
0.92 and TRUNK builds on Jenkins have been unstable lately. I think we
shouldn't make a successful build further.

Cheers

On Thu, Nov 17, 2011 at 10:47 AM, Gary Helmling <[EMAIL PROTECTED]> wrote:

> Yes, the ZK authentication in 2418 is the final piece of the puzzle.
>
>
> On Thu, Nov 17, 2011 at 10:44 AM, Stack <[EMAIL PROTECTED]> wrote:
> > On Thu, Nov 17, 2011 at 10:33 AM, Gary Helmling <[EMAIL PROTECTED]>
> wrote:
> >> Hi all,
> >>
> >> I intend to commit the following patches for HBase security to the
> >> 0.92 branch and trunk at the end of day today:
> >>
> >> HBASE-2742: Provide a secure RPC engine for HBase
> >> https://reviews.apache.org/r/1991/
> >>
> >> HBASE-3025: Coprocessor based simple access control
> >> https://reviews.apache.org/r/2041/
> >>
> >> If you have not reviewed and want to, or feel like you need more time,
> >> please take a look today or speak up now.
> >>
> >
> > Am reviewing now.
> >
> > That would leave Andrew's secure zk issue to complete our security story?
> >
> > St.Ack
> >
>

Re: Security patches: HBASE-2742 and HBASE-3025

Andrew Purtell 2011-11-17, 21:31

Hi Ted,

Integration of security is holding up an 0.92 RC.

I'd like clean Hudson reports too, but that seems thin justification.

Best regards,
- Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
----- Original Message -----
> From: Ted Yu <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]
> Cc:
> Sent: Thursday, November 17, 2011 10:58 AM
> Subject: Re: Security patches: HBASE-2742 and HBASE-3025
>
> I am still going over changes in 2742. But I don't expect to find much.
>
> 2742 and 3025 haven't gone through HadoopQA yet.
> 0.92 and TRUNK builds on Jenkins have been unstable lately. I think we
> shouldn't make a successful build further.
>
> Cheers
>
> On Thu, Nov 17, 2011 at 10:47 AM, Gary Helmling <[EMAIL PROTECTED]>
> wrote:
>
>> Yes, the ZK authentication in 2418 is the final piece of the puzzle.
>>
>>
>> On Thu, Nov 17, 2011 at 10:44 AM, Stack <[EMAIL PROTECTED]> wrote:
>> > On Thu, Nov 17, 2011 at 10:33 AM, Gary Helmling
> <[EMAIL PROTECTED]>
>> wrote:
>> >> Hi all,
>> >>
>> >> I intend to commit the following patches for HBase security to the
>> >> 0.92 branch and trunk at the end of day today:
>> >>
>> >> HBASE-2742: Provide a secure RPC engine for HBase
>> >> https://reviews.apache.org/r/1991/
>> >>
>> >> HBASE-3025: Coprocessor based simple access control
>> >> https://reviews.apache.org/r/2041/
>> >>
>> >> If you have not reviewed and want to, or feel like you need more
> time,
>> >> please take a look today or speak up now.
>> >>
>> >
>> > Am reviewing now.
>> >
>> > That would leave Andrew's secure zk issue to complete our security
> story?
>> >
>> > St.Ack
>> >
>>
>

Re: Security patches: HBASE-2742 and HBASE-3025

Ted Yu 2011-11-17, 21:38

My assumption is that 0.92 RC0 would be released after:
1. 2742 and 3025 get checked in
2. we have a good Jenkins build after that

We can iron out build issues between #1 and #2 above.
HadoopQA would make this easier by telling us which tests may fail.

This is my personal opinion.

On Thu, Nov 17, 2011 at 1:31 PM, Andrew Purtell <[EMAIL PROTECTED]> wrote:

> Hi Ted,
>
> Integration of security is holding up an 0.92 RC.
>
> I'd like clean Hudson reports too, but that seems thin justification.
>
> Best regards,
>
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>
>
> ----- Original Message -----
> > From: Ted Yu <[EMAIL PROTECTED]>
> > To: [EMAIL PROTECTED]
> > Cc:
> > Sent: Thursday, November 17, 2011 10:58 AM
> > Subject: Re: Security patches: HBASE-2742 and HBASE-3025
> >
> > I am still going over changes in 2742. But I don't expect to find much.
> >
> > 2742 and 3025 haven't gone through HadoopQA yet.
> > 0.92 and TRUNK builds on Jenkins have been unstable lately. I think we
> > shouldn't make a successful build further.
> >
> > Cheers
> >
> > On Thu, Nov 17, 2011 at 10:47 AM, Gary Helmling <[EMAIL PROTECTED]>
> > wrote:
> >
> >> Yes, the ZK authentication in 2418 is the final piece of the puzzle.
> >>
> >>
> >> On Thu, Nov 17, 2011 at 10:44 AM, Stack <[EMAIL PROTECTED]> wrote:
> >> > On Thu, Nov 17, 2011 at 10:33 AM, Gary Helmling
> > <[EMAIL PROTECTED]>
> >> wrote:
> >> >> Hi all,
> >> >>
> >> >> I intend to commit the following patches for HBase security to the
> >> >> 0.92 branch and trunk at the end of day today:
> >> >>
> >> >> HBASE-2742: Provide a secure RPC engine for HBase
> >> >> https://reviews.apache.org/r/1991/
> >> >>
> >> >> HBASE-3025: Coprocessor based simple access control
> >> >> https://reviews.apache.org/r/2041/
> >> >>
> >> >> If you have not reviewed and want to, or feel like you need more
> > time,
> >> >> please take a look today or speak up now.
> >> >>
> >> >
> >> > Am reviewing now.
> >> >
> >> > That would leave Andrew's secure zk issue to complete our security
> > story?
> >> >
> >> > St.Ack
> >> >
> >>
> >
>

Re: Security patches: HBASE-2742 and HBASE-3025

Andrew Purtell 2011-11-17, 21:50

Also 2418 needs to go in.

There are three patches:

- HBASE-2742: Secure RPC engine

- HBASE-3025: Access controller

- HBASE-2418: Prevents subversion of 3025 via direct ZK client access.

I have no objection to using HadoopQA.

>________________________________
>From: Ted Yu <[EMAIL PROTECTED]>
>To: [EMAIL PROTECTED]; Andrew Purtell <[EMAIL PROTECTED]>
>Sent: Thursday, November 17, 2011 1:38 PM
>Subject: Re: Security patches: HBASE-2742 and HBASE-3025
>
>
>My assumption is that 0.92 RC0 would be released after:
>1. 2742 and 3025 get checked in
>2. we have a good Jenkins build after that
>
>We can iron out build issues between #1 and #2 above.
>HadoopQA would make this easier by telling us which tests may fail.
>
>This is my personal opinion.
>
>
>On Thu, Nov 17, 2011 at 1:31 PM, Andrew Purtell <[EMAIL PROTECTED]> wrote:
>
>Hi Ted,
>>
>>Integration of security is holding up an 0.92 RC.
>>
>>I'd like clean Hudson reports too, but that seems thin justification.
>>
>>Best regards,
>>
>>
>> - Andy
>>
>>Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
>>
>>
>>
>>----- Original Message -----
>>> From: Ted Yu <[EMAIL PROTECTED]>
>>> To: [EMAIL PROTECTED]
>>> Cc:
>>> Sent: Thursday, November 17, 2011 10:58 AM
>>> Subject: Re: Security patches: HBASE-2742 and HBASE-3025
>>>
>>> I am still going over changes in 2742. But I don't expect to find much.
>>>
>>> 2742 and 3025 haven't gone through HadoopQA yet.
>>> 0.92 and TRUNK builds on Jenkins have been unstable lately. I think we
>>> shouldn't make a successful build further.
>>>
>>> Cheers
>>>
>>> On Thu, Nov 17, 2011 at 10:47 AM, Gary Helmling <[EMAIL PROTECTED]>
>>> wrote:
>>>
>>>> Yes, the ZK authentication in 2418 is the final piece of the puzzle.
>>>>
>>>>
>>>> On Thu, Nov 17, 2011 at 10:44 AM, Stack <[EMAIL PROTECTED]> wrote:
>>>> > On Thu, Nov 17, 2011 at 10:33 AM, Gary Helmling
>>> <[EMAIL PROTECTED]>
>>>> wrote:
>>>> >> Hi all,
>>>> >>
>>>> >> I intend to commit the following patches for HBase security to the
>>>> >> 0.92 branch and trunk at the end of day today:
>>>> >>
>>>> >> HBASE-2742: Provide a secure RPC engine for HBase
>>>> >> https://reviews.apache.org/r/1991/
>>>> >>
>>>> >> HBASE-3025: Coprocessor based simple access control
>>>> >> https://reviews.apache.org/r/2041/
>>>> >>
>>>> >> If you have not reviewed and want to, or feel like you need more
>>> time,
>>>> >> please take a look today or speak up now.
>>>> >>
>>>> >
>>>> > Am reviewing now.
>>>> >
>>>> > That would leave Andrew's secure zk issue to complete our security
>>> story?
>>>> >
>>>> > St.Ack
>>>> >
>>>>
>>>
>>
>
>
>

Re: Security patches: HBASE-2742 and HBASE-3025

Gary Helmling 2011-11-19, 00:58

HBASE-2742 and HBASE-3025 have both been committed to 0.92 and trunk.

Now that we have those and the "security" profile in place, we need to
setup additional Jenkins builds using the security profile, one for
0.92 and one for trunk.

These would be the same as the existing builds, just with the addition
of "-P security" to the build command.

Once we get to maven modules, security can just be part of the normal
build, running it's own subset of tests, with some additional IPC
related tests for exercising SecureRpcEngine.

--gh
On Thu, Nov 17, 2011 at 1:50 PM, Andrew Purtell <[EMAIL PROTECTED]> wrote:
> Also 2418 needs to go in.
>
> There are three patches:
>
> - HBASE-2742: Secure RPC engine
>
> - HBASE-3025: Access controller
>
> - HBASE-2418: Prevents subversion of 3025 via direct ZK client access.
>
> I have no objection to using HadoopQA.
>
>
>
>>________________________________
>>From: Ted Yu <[EMAIL PROTECTED]>
>>To: [EMAIL PROTECTED]; Andrew Purtell <[EMAIL PROTECTED]>
>>Sent: Thursday, November 17, 2011 1:38 PM
>>Subject: Re: Security patches: HBASE-2742 and HBASE-3025
>>
>>
>>My assumption is that 0.92 RC0 would be released after:
>>1. 2742 and 3025 get checked in
>>2. we have a good Jenkins build after that
>>
>>We can iron out build issues between #1 and #2 above.
>>HadoopQA would make this easier by telling us which tests may fail.
>>
>>This is my personal opinion.
>>
>>
>>On Thu, Nov 17, 2011 at 1:31 PM, Andrew Purtell <[EMAIL PROTECTED]> wrote:
>>
>>Hi Ted,
>>>
>>>Integration of security is holding up an 0.92 RC.
>>>
>>>I'd like clean Hudson reports too, but that seems thin justification.
>>>
>>>Best regards,
>>>
>>>
>>> - Andy
>>>
>>>Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
>>>
>>>
>>>
>>>----- Original Message -----
>>>> From: Ted Yu <[EMAIL PROTECTED]>
>>>> To: [EMAIL PROTECTED]
>>>> Cc:
>>>> Sent: Thursday, November 17, 2011 10:58 AM
>>>> Subject: Re: Security patches: HBASE-2742 and HBASE-3025
>>>>
>>>> I am still going over changes in 2742. But I don't expect to find much.
>>>>
>>>> 2742 and 3025 haven't gone through HadoopQA yet.
>>>> 0.92 and TRUNK builds on Jenkins have been unstable lately. I think we
>>>> shouldn't make a successful build further.
>>>>
>>>> Cheers
>>>>
>>>> On Thu, Nov 17, 2011 at 10:47 AM, Gary Helmling <[EMAIL PROTECTED]>
>>>> wrote:
>>>>
>>>>> Yes, the ZK authentication in 2418 is the final piece of the puzzle.
>>>>>
>>>>>
>>>>> On Thu, Nov 17, 2011 at 10:44 AM, Stack <[EMAIL PROTECTED]> wrote:
>>>>> > On Thu, Nov 17, 2011 at 10:33 AM, Gary Helmling
>>>> <[EMAIL PROTECTED]>
>>>>> wrote:
>>>>> >> Hi all,
>>>>> >>
>>>>> >> I intend to commit the following patches for HBase security to the
>>>>> >> 0.92 branch and trunk at the end of day today:
>>>>> >>
>>>>> >> HBASE-2742: Provide a secure RPC engine for HBase
>>>>> >> https://reviews.apache.org/r/1991/
>>>>> >>
>>>>> >> HBASE-3025: Coprocessor based simple access control
>>>>> >> https://reviews.apache.org/r/2041/
>>>>> >>
>>>>> >> If you have not reviewed and want to, or feel like you need more
>>>> time,
>>>>> >> please take a look today or speak up now.
>>>>> >>
>>>>> >
>>>>> > Am reviewing now.
>>>>> >
>>>>> > That would leave Andrew's secure zk issue to complete our security
>>>> story?
>>>>> >
>>>>> > St.Ack
>>>>> >
>>>>>
>>>>
>>>
>>
>>
>>

Re: Security patches: HBASE-2742 and HBASE-3025

Stack 2011-11-19, 23:21

On Sat, Nov 19, 2011 at 3:17 PM, Stack <[EMAIL PROTECTED]> wrote:
> On Fri, Nov 18, 2011 at 4:58 PM, Gary Helmling <[EMAIL PROTECTED]> wrote:
>
>> Now that we have those and the "security" profile in place, we need to
>> setup additional Jenkins builds using the security profile, one for
>> 0.92 and one for trunk.
>>
>> These would be the same as the existing builds, just with the addition
>> of "-P security" to the build command.
>>
>
> I added them building once a day each (if changes).
> St.Ack
>

What should I put up for an RC? Two tarballs? One for secure and
another for insecure? (Trying it to see what happens).

St.Ack