There's just a few issues left.  Check it out:
https://issues.apache.org/jira/browse/HBASE/fixforversion/12314223

I'm hoping we can kill these last few and post an RC this week.  If
there is anything you crew could do to help along the RC, I would
appreciate the hand (Or, if there is something you think has to make
it into 0.92.0, please speak up).

Thanks all,
St.Ack

On Mon, Oct 31, 2011 at 11:16 AM, Stack <[EMAIL PROTECTED]> wrote:
> There's just a few issues left.  Check it out:
> https://issues.apache.org/jira/browse/HBASE/fixforversion/12314223
>
> I'm hoping we can kill these last few and post an RC this week.  If
> there is anything you crew could do to help along the RC, I would
> appreciate the hand (Or, if there is something you think has to make
> it into 0.92.0, please speak up).

That's great news! If you guys are interested in integration testing of this RC
I can offer the Hadoop-0.22 release that Konstantin, Cos and I are currently
working on as part of the Bigtop integration testing. The Hadoop stack there
includes HBase 0.92 on top of HDFS 0.22.

Thanks,
Roman.

On Mon, Oct 31, 2011 at 11:19 AM, Roman Shaposhnik <[EMAIL PROTECTED]> wrote:
> That's great news! If you guys are interested in integration testing of this RC
> I can offer the Hadoop-0.22 release that Konstantin, Cos and I are currently
> working on as part of the Bigtop integration testing. The Hadoop stack there
> includes HBase 0.92 on top of HDFS 0.22.
>

For sure I'm interested.  Would you have time for me some time this
week or after HW to work on this together?
St.Ack

https://builds.apache.org/job/HBase-0.92/91 passed

Shall we target Friday for RC0 ?

On Mon, Oct 31, 2011 at 11:19 AM, Roman Shaposhnik <[EMAIL PROTECTED]> wrote:

> On Mon, Oct 31, 2011 at 11:16 AM, Stack <[EMAIL PROTECTED]> wrote:
> > There's just a few issues left.  Check it out:
> > https://issues.apache.org/jira/browse/HBASE/fixforversion/12314223
> >
> > I'm hoping we can kill these last few and post an RC this week.  If
> > there is anything you crew could do to help along the RC, I would
> > appreciate the hand (Or, if there is something you think has to make
> > it into 0.92.0, please speak up).
>
> That's great news! If you guys are interested in integration testing of
> this RC
> I can offer the Hadoop-0.22 release that Konstantin, Cos and I are
> currently
> working on as part of the Bigtop integration testing. The Hadoop stack
> there
> includes HBase 0.92 on top of HDFS 0.22.
>
> Thanks,
> Roman.
>

On Mon, Oct 31, 2011 at 9:28 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
> https://builds.apache.org/job/HBase-0.92/91 passed
>
> Shall we target Friday for RC0 ?
>

Yes.  I hope to get an RC up this week.
St.Ack

Hi Stack,

I sent you an email after our meetings last week.  I am not sure if my
email was received.  Here are a couple issues that would improve
Hadoop 0.20.205.0 and HBase 0.92 integration.

HBASE-4415
HBASE-4523
HBASE-4535
HBASE-4635
HBASE-4498

We are hopping if those issues can be committed before RC0?  Thanks

regards,
Eric

On Mon, Oct 31, 2011 at 10:42 PM, Stack <[EMAIL PROTECTED]> wrote:
> On Mon, Oct 31, 2011 at 9:28 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>> https://builds.apache.org/job/HBase-0.92/91 passed
>>
>> Shall we target Friday for RC0 ?
>>
>
> Yes.  I hope to get an RC up this week.
> St.Ack
>

I'd like to lobby for the security patches (HBASE-2742 and HBASE-3025)
going in to 0.92.  This may sound like a big change at this stage, but
in reality the impact on current HBase core is pretty contained:

* HBASE-2742 adds a "security" profile to the build, which only
includes the security sources (security/src/...) if enabled.  The
changes to core code in this patch mostly just enable subclassing of
HBaseClient and HBaseServer for implementation of the SecureRpcEngine.

* HBASE-3025 adds classes to the "security" source tree to implement
authorization checking.  These break down into:
  - an AccessController coprocessor implementation for authorization
checking.  If not enabled, this code has no impact on core.
  - additional shell commands for ACL manipulation: grant, revoke,
user_permission.  These implementations check for existence of the
security classes on the classpath and will fail gracefully with an
error message that security is not available.

The use of a maven profile means that security components will only be
included in the build when activated (mvn ... -P security), so we can
continue to do builds and releases without the security components, if
needed.  To start with, we may want to do separate release packages
for the base HBase version vs. HBase with security (0.92.0 and
0.92.0-security), so that only those interested in using the security
features need be impacted by them.

At Trend, we've been running a release with HBase security for the
past 6 months in production, so it's been proven to work for us.  I
know that there are other groups interested in working with HBase
security on 0.92, so I would really like to find a way meet that need.
 At the same time, all of the security components have been built to
be optionally enabled, so I think there is very little risk to core by
including it.

--gh

On Mon, Oct 31, 2011 at 11:16 AM, Stack <[EMAIL PROTECTED]> wrote:
> There's just a few issues left.  Check it out:
> https://issues.apache.org/jira/browse/HBASE/fixforversion/12314223
>
> I'm hoping we can kill these last few and post an RC this week.  If
> there is anything you crew could do to help along the RC, I would
> appreciate the hand (Or, if there is something you think has to make
> it into 0.92.0, please speak up).
>
> Thanks all,
> St.Ack
>

On Tue, Nov 1, 2011 at 12:00 PM, Gary Helmling <[EMAIL PROTECTED]> wrote:
> I'd like to lobby for the security patches (HBASE-2742 and HBASE-3025)
> going in to 0.92.  This may sound like a big change at this stage, but
> in reality the impact on current HBase core is pretty contained:
>

I've not read the patches but +1 on trying to get them in.
St.Ack

> From: Gary Helmling <[EMAIL PROTECTED]>
> I'd like to lobby for the security patches (HBASE-2742 and HBASE-3025)
> going in to 0.92.  This may sound like a big change at this stage, but
> in reality the impact on current HBase core is pretty contained

[...]
> At Trend, we've been running a release with HBase security for the
> past 6 months in production, so it's been proven to work for us.  I
> know that there are other groups interested in working with HBase
> security on 0.92, so I would really like to find a way meet that need.
> At the same time, all of the security components have been built to
> be optionally enabled, so I think there is very little risk to core by
> including it.
FYI, There is also one more patch forthcoming which adds integration for secure ZooKeeper (3.4 RC), if available.
+1 
We've engineered these features to have minimal impact on core. Security is implemented as Coprocessors. If you don't want it, just don't load them. Zero impact in that case. Also, it is possible (though strange) to enable security on some tables, but not on others.

Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)

+1. That would be a good milestone for 0.92.
I also wish we'd get HBASE-4344 and related jiras (HBASE-2856, etc) in.
-- Lars
________________________________
From: Gary Helmling <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]
Sent: Tuesday, November 1, 2011 12:00 PM
Subject: Re: An 0.92.0 Release Candidate this week?

I'd like to lobby for the security patches (HBASE-2742 and HBASE-3025)
going in to 0.92.  This may sound like a big change at this stage, but
in reality the impact on current HBase core is pretty contained:

* HBASE-2742 adds a "security" profile to the build, which only
includes the security sources (security/src/...) if enabled.  The
changes to core code in this patch mostly just enable subclassing of
HBaseClient and HBaseServer for implementation of the SecureRpcEngine.

* HBASE-3025 adds classes to the "security" source tree to implement
authorization checking.  These break down into:
  - an AccessController coprocessor implementation for authorization
checking.  If not enabled, this code has no impact on core.
  - additional shell commands for ACL manipulation: grant, revoke,
user_permission.  These implementations check for existence of the
security classes on the classpath and will fail gracefully with an
error message that security is not available.

The use of a maven profile means that security components will only be
included in the build when activated (mvn ... -P security), so we can
continue to do builds and releases without the security components, if
needed.  To start with, we may want to do separate release packages
for the base HBase version vs. HBase with security (0.92.0 and
0.92.0-security), so that only those interested in using the security
features need be impacted by them.

At Trend, we've been running a release with HBase security for the
past 6 months in production, so it's been proven to work for us.  I
know that there are other groups interested in working with HBase
security on 0.92, so I would really like to find a way meet that need.
At the same time, all of the security components have been built to
be optionally enabled, so I think there is very little risk to core by
including it.

--gh

On Mon, Oct 31, 2011 at 11:16 AM, Stack <[EMAIL PROTECTED]> wrote:
> There's just a few issues left.  Check it out:
> https://issues.apache.org/jira/browse/HBASE/fixforversion/12314223
>
> I'm hoping we can kill these last few and post an RC this week.  If
> there is anything you crew could do to help along the RC, I would
> appreciate the hand (Or, if there is something you think has to make
> it into 0.92.0, please speak up).
>
> Thanks all,
> St.Ack
>

HBASE-2856 is harder than initially expected.

It is marked for HBase 0.94.

On Tue, Nov 1, 2011 at 2:24 PM, lars hofhansl <[EMAIL PROTECTED]> wrote:

> +1. That would be a good milestone for 0.92.
>
>
> I also wish we'd get HBASE-4344 and related jiras (HBASE-2856, etc) in.
>
>
> -- Lars
> ________________________________
> From: Gary Helmling <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]
> Sent: Tuesday, November 1, 2011 12:00 PM
> Subject: Re: An 0.92.0 Release Candidate this week?
>
> I'd like to lobby for the security patches (HBASE-2742 and HBASE-3025)
> going in to 0.92.  This may sound like a big change at this stage, but
> in reality the impact on current HBase core is pretty contained:
>
> * HBASE-2742 adds a "security" profile to the build, which only
> includes the security sources (security/src/...) if enabled.  The
> changes to core code in this patch mostly just enable subclassing of
> HBaseClient and HBaseServer for implementation of the SecureRpcEngine.
>
> * HBASE-3025 adds classes to the "security" source tree to implement
> authorization checking.  These break down into:
>   - an AccessController coprocessor implementation for authorization
> checking.  If not enabled, this code has no impact on core.
>   - additional shell commands for ACL manipulation: grant, revoke,
> user_permission.  These implementations check for existence of the
> security classes on the classpath and will fail gracefully with an
> error message that security is not available.
>
> The use of a maven profile means that security components will only be
> included in the build when activated (mvn ... -P security), so we can
> continue to do builds and releases without the security components, if
> needed.  To start with, we may want to do separate release packages
> for the base HBase version vs. HBase with security (0.92.0 and
> 0.92.0-security), so that only those interested in using the security
> features need be impacted by them.
>
> At Trend, we've been running a release with HBase security for the
> past 6 months in production, so it's been proven to work for us.  I
> know that there are other groups interested in working with HBase
> security on 0.92, so I would really like to find a way meet that need.
> At the same time, all of the security components have been built to
> be optionally enabled, so I think there is very little risk to core by
> including it.
>
> --gh
>
> On Mon, Oct 31, 2011 at 11:16 AM, Stack <[EMAIL PROTECTED]> wrote:
> > There's just a few issues left.  Check it out:
> > https://issues.apache.org/jira/browse/HBASE/fixforversion/12314223
> >
> > I'm hoping we can kill these last few and post an RC this week.  If
> > there is anything you crew could do to help along the RC, I would
> > appreciate the hand (Or, if there is something you think has to make
> > it into 0.92.0, please speak up).
> >
> > Thanks all,
> > St.Ack
> >
>

Regarding HBASE-4415 and related, it makes me nervous to add it in
just before we cut the RC but at the same time I feel like I don't
fully understand what it's doing. Are we adding a new mandatory
configuration step?

Thx for clarifying.

J-D

On Tue, Nov 1, 2011 at 9:52 AM, Eric Yang <[EMAIL PROTECTED]> wrote:
> Hi Stack,
>
> I sent you an email after our meetings last week.  I am not sure if my
> email was received.  Here are a couple issues that would improve
> Hadoop 0.20.205.0 and HBase 0.92 integration.
>
> HBASE-4415
> HBASE-4523
> HBASE-4535
> HBASE-4635
> HBASE-4498
>
> We are hopping if those issues can be committed before RC0?  Thanks
>
> regards,
> Eric
>
> On Mon, Oct 31, 2011 at 10:42 PM, Stack <[EMAIL PROTECTED]> wrote:
>> On Mon, Oct 31, 2011 at 9:28 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>>> https://builds.apache.org/job/HBase-0.92/91 passed
>>>
>>> Shall we target Friday for RC0 ?
>>>
>>
>> Yes.  I hope to get an RC up this week.
>> St.Ack
>>
>

For improving installation experience for HBase 0.92 with Hadoop 0.20.205,  I wish we could get these patches
into 0.92.0 RC:

HBASE-4415 Add configuration script for setup HBase (hbase-setup-conf.sh)
HBASE-4523 dfs.support.append config should be present in the hadoop configs, we should remove them from hbase so the user is not confused when they see the config in 2 places
HBASE-4535 hbase-env.sh in hbase rpm does not set HBASE_CONF_DIR
HBASE-4635 Remove dependency of java for rpm/deb packaging
HBASE-4498 HBase RPM/DEB packages attempt to setup ZooKeeper environment incorrectly

Patches are available for all jiras and reviewed.

regards,
Eric

On Nov 1, 2011, at 2:53 PM, Ted Yu wrote:

> HBASE-2856 is harder than initially expected.
>
> It is marked for HBase 0.94.
>
> On Tue, Nov 1, 2011 at 2:24 PM, lars hofhansl <[EMAIL PROTECTED]> wrote:
>
>> +1. That would be a good milestone for 0.92.
>>
>>
>> I also wish we'd get HBASE-4344 and related jiras (HBASE-2856, etc) in.
>>
>>
>> -- Lars
>> ________________________________
>> From: Gary Helmling <[EMAIL PROTECTED]>
>> To: [EMAIL PROTECTED]
>> Sent: Tuesday, November 1, 2011 12:00 PM
>> Subject: Re: An 0.92.0 Release Candidate this week?
>>
>> I'd like to lobby for the security patches (HBASE-2742 and HBASE-3025)
>> going in to 0.92.  This may sound like a big change at this stage, but
>> in reality the impact on current HBase core is pretty contained:
>>
>> * HBASE-2742 adds a "security" profile to the build, which only
>> includes the security sources (security/src/...) if enabled.  The
>> changes to core code in this patch mostly just enable subclassing of
>> HBaseClient and HBaseServer for implementation of the SecureRpcEngine.
>>
>> * HBASE-3025 adds classes to the "security" source tree to implement
>> authorization checking.  These break down into:
>>  - an AccessController coprocessor implementation for authorization
>> checking.  If not enabled, this code has no impact on core.
>>  - additional shell commands for ACL manipulation: grant, revoke,
>> user_permission.  These implementations check for existence of the
>> security classes on the classpath and will fail gracefully with an
>> error message that security is not available.
>>
>> The use of a maven profile means that security components will only be
>> included in the build when activated (mvn ... -P security), so we can
>> continue to do builds and releases without the security components, if
>> needed.  To start with, we may want to do separate release packages
>> for the base HBase version vs. HBase with security (0.92.0 and
>> 0.92.0-security), so that only those interested in using the security
>> features need be impacted by them.
>>
>> At Trend, we've been running a release with HBase security for the
>> past 6 months in production, so it's been proven to work for us.  I
>> know that there are other groups interested in working with HBase
>> security on 0.92, so I would really like to find a way meet that need.
>> At the same time, all of the security components have been built to
>> be optionally enabled, so I think there is very little risk to core by
>> including it.
>>
>> --gh
>>
>> On Mon, Oct 31, 2011 at 11:16 AM, Stack <[EMAIL PROTECTED]> wrote:
>>> There's just a few issues left.  Check it out:
>>> https://issues.apache.org/jira/browse/HBASE/fixforversion/12314223
>>>
>>> I'm hoping we can kill these last few and post an RC this week.  If
>>> there is anything you crew could do to help along the RC, I would
>>> appreciate the hand (Or, if there is something you think has to make
>>> it into 0.92.0, please speak up).
>>>
>>> Thanks all,
>>> St.Ack
>>>
>>

On Mon, Oct 31, 2011 at 11:16 AM, Stack <[EMAIL PROTECTED]> wrote:
> There's just a few issues left.  Check it out:
> https://issues.apache.org/jira/browse/HBASE/fixforversion/12314223
>
> I'm hoping we can kill these last few and post an RC this week.  If
> there is anything you crew could do to help along the RC, I would
> appreciate the hand (Or, if there is something you think has to make
> it into 0.92.0, please speak up).

I know this is trivial, but can we also add this one to the list:
    https://issues.apache.org/jira/browse/HBASE-4719

I can supply the patch by end of day tomorrow.

Thanks,
Roman.

The configuration script is useful tool to generate configuration.  The recommended script is creating a pattern for user to manage configuration consistently across Hadoop related projects.
It is optional to use the configuration scripts, but it is a good starting point to show people how to configure a secure cluster.

regards,
Eric

On Nov 1, 2011, at 5:17 PM, Jean-Daniel Cryans wrote:

> Regarding HBASE-4415 and related, it makes me nervous to add it in
> just before we cut the RC but at the same time I feel like I don't
> fully understand what it's doing. Are we adding a new mandatory
> configuration step?
>
> Thx for clarifying.
>
> J-D
>
> On Tue, Nov 1, 2011 at 9:52 AM, Eric Yang <[EMAIL PROTECTED]> wrote:
>> Hi Stack,
>>
>> I sent you an email after our meetings last week.  I am not sure if my
>> email was received.  Here are a couple issues that would improve
>> Hadoop 0.20.205.0 and HBase 0.92 integration.
>>
>> HBASE-4415
>> HBASE-4523
>> HBASE-4535
>> HBASE-4635
>> HBASE-4498
>>
>> We are hopping if those issues can be committed before RC0?  Thanks
>>
>> regards,
>> Eric
>>
>> On Mon, Oct 31, 2011 at 10:42 PM, Stack <[EMAIL PROTECTED]> wrote:
>>> On Mon, Oct 31, 2011 at 9:28 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>>>> https://builds.apache.org/job/HBase-0.92/91 passed
>>>>
>>>> Shall we target Friday for RC0 ?
>>>>
>>>
>>> Yes.  I hope to get an RC up this week.
>>> St.Ack
>>>
>>

Eric,

> It is optional to use the configuration scripts, but it is a good starting point 
> to show people how to configure a secure cluster.
As I mentioned on 4415, the script as is won't set up "secure HBase" as it will be defined beyond just HDFS level stuff. I think it is premature.

Best regards,
   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
----- Original Message -----
> From: Eric Yang <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]
> Cc:
> Sent: Tuesday, November 1, 2011 5:25 PM
> Subject: Re: An 0.92.0 Release Candidate this week?
>
>T he configuration script is useful tool to generate configuration.  The
> recommended script is creating a pattern for user to manage configuration
> consistently across Hadoop related projects.
> It is optional to use the configuration scripts, but it is a good starting point
> to show people how to configure a secure cluster.
>
> regards,
> Eric
>
> On Nov 1, 2011, at 5:17 PM, Jean-Daniel Cryans wrote:
>
>>  Regarding HBASE-4415 and related, it makes me nervous to add it in
>>  just before we cut the RC but at the same time I feel like I don't
>>  fully understand what it's doing. Are we adding a new mandatory
>>  configuration step?
>>
>>  Thx for clarifying.
>>
>>  J-D
>>
>>  On Tue, Nov 1, 2011 at 9:52 AM, Eric Yang <[EMAIL PROTECTED]>
> wrote:
>>>  Hi Stack,
>>>
>>>  I sent you an email after our meetings last week.  I am not sure if my
>>>  email was received.  Here are a couple issues that would improve
>>>  Hadoop 0.20.205.0 and HBase 0.92 integration.
>>>
>>>  HBASE-4415
>>>  HBASE-4523
>>>  HBASE-4535
>>>  HBASE-4635
>>>  HBASE-4498
>>>
>>>  We are hopping if those issues can be committed before RC0?  Thanks
>>>
>>>  regards,
>>>  Eric
>>>
>>>  On Mon, Oct 31, 2011 at 10:42 PM, Stack <[EMAIL PROTECTED]> wrote:
>>>>  On Mon, Oct 31, 2011 at 9:28 PM, Ted Yu <[EMAIL PROTECTED]>
> wrote:
>>>>>  https://builds.apache.org/job/HBase-0.92/91 passed
>>>>>
>>>>>  Shall we target Friday for RC0 ?
>>>>>
>>>>
>>>>  Yes.  I hope to get an RC up this week.
>>>>  St.Ack
>>>>
>>>
>

Hi Andrew,

The patch does not advertise Secure HBase, but provides ability to setup non-secure HBase on Secure HDFS.  It would be a release note documentation process to set the right expectation.  Secure HBase features can be added separate from setup scripts.  Does this sound like reasonable?  Once HBASE-2742 and HBASE-3025 are committed.  Additional knobs can be added to the setup script in separate jiras.  Would this work?  It would be helpful to keep evolution progressing than patch left out and become out of date.

regards,
Eric

On Nov 1, 2011, at 5:46 PM, Andrew Purtell wrote:

> Eric,
>
>> It is optional to use the configuration scripts, but it is a good starting point
>> to show people how to configure a secure cluster.
>
>
> As I mentioned on 4415, the script as is won't set up "secure HBase" as it will be defined beyond just HDFS level stuff. I think it is premature.
>
> Best regards,
>
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
>
>
> ----- Original Message -----
>> From: Eric Yang <[EMAIL PROTECTED]>
>> To: [EMAIL PROTECTED]
>> Cc:
>> Sent: Tuesday, November 1, 2011 5:25 PM
>> Subject: Re: An 0.92.0 Release Candidate this week?
>>
>> T he configuration script is useful tool to generate configuration.  The
>> recommended script is creating a pattern for user to manage configuration
>> consistently across Hadoop related projects.
>> It is optional to use the configuration scripts, but it is a good starting point
>> to show people how to configure a secure cluster.
>>
>> regards,
>> Eric
>>
>> On Nov 1, 2011, at 5:17 PM, Jean-Daniel Cryans wrote:
>>
>>> Regarding HBASE-4415 and related, it makes me nervous to add it in
>>> just before we cut the RC but at the same time I feel like I don't
>>> fully understand what it's doing. Are we adding a new mandatory
>>> configuration step?
>>>
>>> Thx for clarifying.
>>>
>>> J-D
>>>
>>> On Tue, Nov 1, 2011 at 9:52 AM, Eric Yang <[EMAIL PROTECTED]>
>> wrote:
>>>> Hi Stack,
>>>>
>>>> I sent you an email after our meetings last week.  I am not sure if my
>>>> email was received.  Here are a couple issues that would improve
>>>> Hadoop 0.20.205.0 and HBase 0.92 integration.
>>>>
>>>> HBASE-4415
>>>> HBASE-4523
>>>> HBASE-4535
>>>> HBASE-4635
>>>> HBASE-4498
>>>>
>>>> We are hopping if those issues can be committed before RC0?  Thanks
>>>>
>>>> regards,
>>>> Eric
>>>>
>>>> On Mon, Oct 31, 2011 at 10:42 PM, Stack <[EMAIL PROTECTED]> wrote:
>>>>> On Mon, Oct 31, 2011 at 9:28 PM, Ted Yu <[EMAIL PROTECTED]>
>> wrote:
>>>>>> https://builds.apache.org/job/HBase-0.92/91 passed
>>>>>>
>>>>>> Shall we target Friday for RC0 ?
>>>>>>
>>>>>
>>>>> Yes.  I hope to get an RC up this week.
>>>>> St.Ack
>>>>>
>>>>
>>

That seems small potatoes Roman.  Add up a patch and we'll commit.
St.Ack

On Tue, Nov 1, 2011 at 5:21 PM, Roman Shaposhnik <[EMAIL PROTECTED]> wrote:
> On Mon, Oct 31, 2011 at 11:16 AM, Stack <[EMAIL PROTECTED]> wrote:
>> There's just a few issues left.  Check it out:
>> https://issues.apache.org/jira/browse/HBASE/fixforversion/12314223
>>
>> I'm hoping we can kill these last few and post an RC this week.  If
>> there is anything you crew could do to help along the RC, I would
>> appreciate the hand (Or, if there is something you think has to make
>> it into 0.92.0, please speak up).
>
> I know this is trivial, but can we also add this one to the list:
>    https://issues.apache.org/jira/browse/HBASE-4719
>
> I can supply the patch by end of day tomorrow.
>
> Thanks,
> Roman.
>

Roman:
Go for it.

Thanks

On Tue, Nov 1, 2011 at 5:21 PM, Roman Shaposhnik <[EMAIL PROTECTED]> wrote:

> On Mon, Oct 31, 2011 at 11:16 AM, Stack <[EMAIL PROTECTED]> wrote:
> > There's just a few issues left.  Check it out:
> > https://issues.apache.org/jira/browse/HBASE/fixforversion/12314223
> >
> > I'm hoping we can kill these last few and post an RC this week.  If
> > there is anything you crew could do to help along the RC, I would
> > appreciate the hand (Or, if there is something you think has to make
> > it into 0.92.0, please speak up).
>
> I know this is trivial, but can we also add this one to the list:
>    https://issues.apache.org/jira/browse/HBASE-4719
>
> I can supply the patch by end of day tomorrow.
>
> Thanks,
> Roman.
>

On Tue, Nov 1, 2011 at 9:52 AM, Eric Yang <[EMAIL PROTECTED]> wrote:
> Hi Stack,
>
> I sent you an email after our meetings last week.  I am not sure if my
> email was received.  Here are a couple issues that would improve
> Hadoop 0.20.205.0 and HBase 0.92 integration.
>
> HBASE-4415
> HBASE-4523
> HBASE-4535
> HBASE-4635
> HBASE-4498
>

Do these not depend on HBASE-4337, which would change the directory
structure under hbase base.dir?

St.Ack

These patches don't depend on the directory structure layout change, but my progress on HBASE-4337 depends on those patches.

regards,
Eric

On Nov 1, 2011, at 8:56 PM, Stack wrote:

> On Tue, Nov 1, 2011 at 9:52 AM, Eric Yang <[EMAIL PROTECTED]> wrote:
>> Hi Stack,
>>
>> I sent you an email after our meetings last week.  I am not sure if my
>> email was received.  Here are a couple issues that would improve
>> Hadoop 0.20.205.0 and HBase 0.92 integration.
>>
>> HBASE-4415
>> HBASE-4523
>> HBASE-4535
>> HBASE-4635
>> HBASE-4498
>>
>
> Do these not depend on HBASE-4337, which would change the directory
> structure under hbase base.dir?
>
> St.Ack

Hi Eric,
> From: Eric Yang <[EMAIL PROTECTED]>
> The patch does not advertise Secure HBase, but provides ability to setup 
> non-secure HBase on Secure HDFS.  It would be a release note documentation 
Yes, as long as the result as-is now is not advertised as a secure system, because it will not be.

Earlier you said:

>>> It is optional to use the configuration scripts, but it is a good 
>>> starting point to show people how to configure a secure cluster.
which was at the heart of my objection, because that may be only true right now for core.

Best regards,
   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
----- Original Message -----
> From: Eric Yang <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]; Andrew Purtell <[EMAIL PROTECTED]>
> Cc:
> Sent: Tuesday, November 1, 2011 6:26 PM
> Subject: Re: An 0.92.0 Release Candidate this week?
>
> Hi Andrew,
>
> The patch does not advertise Secure HBase, but provides ability to setup
> non-secure HBase on Secure HDFS.  It would be a release note documentation
> process to set the right expectation.  Secure HBase features can be added
> separate from setup scripts.  Does this sound like reasonable?  Once HBASE-2742
> and HBASE-3025 are committed.  Additional knobs can be added to the setup script
> in separate jiras.  Would this work?  It would be helpful to keep evolution
> progressing than patch left out and become out of date.
>
> regards,
> Eric
>
> On Nov 1, 2011, at 5:46 PM, Andrew Purtell wrote:
>
>>  Eric,
>>
>>>  It is optional to use the configuration scripts, but it is a good
> starting point
>>>  to show people how to configure a secure cluster.
>>
>>
>>  As I mentioned on 4415, the script as is won't set up "secure
> HBase" as it will be defined beyond just HDFS level stuff. I think it is
> premature.
>>
>>  Best regards,
>>
>>
>>     - Andy
>>
>>  Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>>
>>
>>  ----- Original Message -----
>>>  From: Eric Yang <[EMAIL PROTECTED]>
>>>  To: [EMAIL PROTECTED]
>>>  Cc:
>>>  Sent: Tuesday, November 1, 2011 5:25 PM
>>>  Subject: Re: An 0.92.0 Release Candidate this week?
>>>
>>>  T he configuration script is useful tool to generate configuration. 
> The
>>>  recommended script is creating a pattern for user to manage
> configuration
>>>  consistently across Hadoop related projects.
>>>  It is optional to use the configuration scripts, but it is a good
> starting point
>>>  to show people how to configure a secure cluster.
>>>
>>>  regards,
>>>  Eric
>>>
>>>  On Nov 1, 2011, at 5:17 PM, Jean-Daniel Cryans wrote:
>>>
>>>>  Regarding HBASE-4415 and related, it makes me nervous to add it in
>>>>  just before we cut the RC but at the same time I feel like I
> don't
>>>>  fully understand what it's doing. Are we adding a new mandatory
>>>>  configuration step?
>>>>
>>>>  Thx for clarifying.
>>>>
>>>>  J-D
>>>>
>>>>  On Tue, Nov 1, 2011 at 9:52 AM, Eric Yang
> <[EMAIL PROTECTED]>
>>>  wrote:
>>>>>  Hi Stack,
>>>>>
>>>>>  I sent you an email after our meetings last week.  I am not
> sure if my
>>>>>  email was received.  Here are a couple issues that would
> improve
>>>>>  Hadoop 0.20.205.0 and HBase 0.92 integration.
>>>>>
>>>>>  HBASE-4415
>>>>>  HBASE-4523
>>>>>  HBASE-4535
>>>>>  HBASE-4635
>>>>>  HBASE-4498
>>>>>
>>>>>  We are hopping if those issues can be committed before RC0? 
> Thanks
>>>>>
>>>>>  regards,
>>>>>  Eric
>>>>>
>>>>>  On Mon, Oct 31, 2011 at 10:42 PM, Stack
> <[EMAIL PROTECTED]> wrote:
>>>>>>  On Mon, Oct 31, 2011 at 9:28 PM, Ted Yu
> <[EMAIL PROTECTED]>
>>>  wrote:
>>>>>>>  https://builds.apache.org/job/HBase-0.92/91 passed
>>>>>>>
>>>>>>>  Shall we target Friday for RC0 ?
>>>>>>>
>>>>>>
>>>>>>  Yes.  I hope to get an RC up this week.
>>>>>>  St.Ack
>>>>>>
>>>>>
>>>
>

On Tue, Nov 1, 2011 at 5:20 PM, Eric Yang <[EMAIL PROTECTED]> wrote:
> For improving installation experience for HBase 0.92 with Hadoop 0.20.205,  I wish we could get these patches
> into 0.92.0 RC:
>
> HBASE-4415 Add configuration script for setup HBase (hbase-setup-conf.sh)
> HBASE-4523 dfs.support.append config should be present in the hadoop configs, we should remove them from hbase so the user is not confused when they see the config in 2 places
> HBASE-4535 hbase-env.sh in hbase rpm does not set HBASE_CONF_DIR
> HBASE-4635 Remove dependency of java for rpm/deb packaging
> HBASE-4498 HBase RPM/DEB packages attempt to setup ZooKeeper environment incorrectly
>
> Patches are available for all jiras and reviewed.
>

These patches look innocuous enough.  If they all go in, what will we
have?  A script that can install and configure an hbase?  We'll have a
new /share dir in our tar ball when its undone.  What else will they
give us?

Thanks Eric,
St.Ack

On Tue, Nov 1, 2011 at 8:51 PM, Stack <[EMAIL PROTECTED]> wrote:
> That seems small potatoes Roman.  Add up a patch and we'll commit.

Thanks! I've uploaded the patch. It was tested in Bigtop against
Hadoop 0.20.205.0
and against 0.22

Thanks,
Roman.

I'd prefer not to make any changes to our existing tarball structure
for the sake of packaging.

I've always believed that packaging should be done external from a
project - this is the successful model used by 99.9% of all packaged
Linux software. Shuffling around our tarball layout for the sake of
packagers is not maintainable, as packages on different
OSes/distributions may have different requirements.

-Todd

On Wed, Nov 2, 2011 at 9:53 AM, Stack <[EMAIL PROTECTED]> wrote:
> On Tue, Nov 1, 2011 at 5:20 PM, Eric Yang <[EMAIL PROTECTED]> wrote:
>> For improving installation experience for HBase 0.92 with Hadoop 0.20.205,  I wish we could get these patches
>> into 0.92.0 RC:
>>
>> HBASE-4415 Add configuration script for setup HBase (hbase-setup-conf.sh)
>> HBASE-4523 dfs.support.append config should be present in the hadoop configs, we should remove them from hbase so the user is not confused when they see the config in 2 places
>> HBASE-4535 hbase-env.sh in hbase rpm does not set HBASE_CONF_DIR
>> HBASE-4635 Remove dependency of java for rpm/deb packaging
>> HBASE-4498 HBase RPM/DEB packages attempt to setup ZooKeeper environment incorrectly
>>
>> Patches are available for all jiras and reviewed.
>>
>
> These patches look innocuous enough.  If they all go in, what will we
> have?  A script that can install and configure an hbase?  We'll have a
> new /share dir in our tar ball when its undone.  What else will they
> give us?
>
> Thanks Eric,
> St.Ack
>

--
Todd Lipcon
Software Engineer, Cloudera

On Wed, Nov 2, 2011 at 9:53 AM, Stack <[EMAIL PROTECTED]> wrote:
>> HBASE-4415 Add configuration script for setup HBase (hbase-setup-conf.sh)
>> HBASE-4523 dfs.support.append config should be present in the hadoop configs, we should remove them from hbase so the user is not confused when they see the config in 2 places
>> HBASE-4535 hbase-env.sh in hbase rpm does not set HBASE_CONF_DIR
>> HBASE-4635 Remove dependency of java for rpm/deb packaging
>> HBASE-4498 HBase RPM/DEB packages attempt to setup ZooKeeper environment incorrectly
>>
>> Patches are available for all jiras and reviewed.
>>
>
> These patches look innocuous enough.  If they all go in, what will we
> have?  A script that can install and configure an hbase?  We'll have a
> new /share dir in our tar ball when its undone.  What else will they
> give us?

Sorry, for perhaps being dense, but where's the patch that would
introduce /share and such?
I'm really nervous about reshuffling the bits at the last moment (if
ever) since I already had
some bad experience with Pig and how Pig/HBase integration  broke after that.

Thanks,
Roman.

Ok.  Good (I asked in the issue if you'd tried it against different
hbase+hadoops -- you can ignore my Q).
St.Ack

On Wed, Nov 2, 2011 at 10:09 AM, Roman Shaposhnik <[EMAIL PROTECTED]> wrote:
> On Tue, Nov 1, 2011 at 8:51 PM, Stack <[EMAIL PROTECTED]> wrote:
>> That seems small potatoes Roman.  Add up a patch and we'll commit.
>
> Thanks! I've uploaded the patch. It was tested in Bigtop against
> Hadoop 0.20.205.0
> and against 0.22
>
> Thanks,
> Roman.
>

Those patches provides:

- A script that can configure HBase.
- Ability to setup unsecure HBase on Secure Hadoop.

Those jiras provide the basics setup which integrate well with other Hadoop components, and I plan to get HBASE-4337 done by tomorrow.

regards,
Eric

On Nov 2, 2011, at 9:53 AM, Stack wrote:

> On Tue, Nov 1, 2011 at 5:20 PM, Eric Yang <[EMAIL PROTECTED]> wrote:
>> For improving installation experience for HBase 0.92 with Hadoop 0.20.205,  I wish we could get these patches
>> into 0.92.0 RC:
>>
>> HBASE-4415 Add configuration script for setup HBase (hbase-setup-conf.sh)
>> HBASE-4523 dfs.support.append config should be present in the hadoop configs, we should remove them from hbase so the user is not confused when they see the config in 2 places
>> HBASE-4535 hbase-env.sh in hbase rpm does not set HBASE_CONF_DIR
>> HBASE-4635 Remove dependency of java for rpm/deb packaging
>> HBASE-4498 HBase RPM/DEB packages attempt to setup ZooKeeper environment incorrectly
>>
>> Patches are available for all jiras and reviewed.
>>
>
> These patches look innocuous enough.  If they all go in, what will we
> have?  A script that can install and configure an hbase?  We'll have a
> new /share dir in our tar ball when its undone.  What else will they
> give us?
>
> Thanks Eric,
> St.Ack

Jira was down last night.  A new patch has been uploaded for HBASE-4337 to match HBase directory layout same as Hadoop directory layout.  The new share directory is undone for the existing directory layout.  The default directory layout is the existing HBase layout.

The binary only layout, and can be build using:

mvn clean package -P binary,deb,rpm -DskipTests

Patch for HBASE-4337 depends on HBASE-4415, 4523, 4635, 4498.
Let me know if there is any questions or concerns.  Thanks

regards,
Eric

On Nov 2, 2011, at 12:08 PM, Eric Yang wrote:

> Those patches provides:
>
> - A script that can configure HBase.
> - Ability to setup unsecure HBase on Secure Hadoop.
>
> Those jiras provide the basics setup which integrate well with other Hadoop components, and I plan to get HBASE-4337 done by tomorrow.
>
> regards,
> Eric
>
> On Nov 2, 2011, at 9:53 AM, Stack wrote:
>
>> On Tue, Nov 1, 2011 at 5:20 PM, Eric Yang <[EMAIL PROTECTED]> wrote:
>>> For improving installation experience for HBase 0.92 with Hadoop 0.20.205,  I wish we could get these patches
>>> into 0.92.0 RC:
>>>
>>> HBASE-4415 Add configuration script for setup HBase (hbase-setup-conf.sh)
>>> HBASE-4523 dfs.support.append config should be present in the hadoop configs, we should remove them from hbase so the user is not confused when they see the config in 2 places
>>> HBASE-4535 hbase-env.sh in hbase rpm does not set HBASE_CONF_DIR
>>> HBASE-4635 Remove dependency of java for rpm/deb packaging
>>> HBASE-4498 HBase RPM/DEB packages attempt to setup ZooKeeper environment incorrectly
>>>
>>> Patches are available for all jiras and reviewed.
>>>
>>
>> These patches look innocuous enough.  If they all go in, what will we
>> have?  A script that can install and configure an hbase?  We'll have a
>> new /share dir in our tar ball when its undone.  What else will they
>> give us?
>>
>> Thanks Eric,
>> St.Ack
>