Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # dev - read short circuit


+
n keywal 2012-09-13, 14:58
+
Jean-Daniel Cryans 2012-09-13, 16:30
+
n keywal 2012-09-13, 16:42
+
Andrew Purtell 2012-09-13, 17:44
+
Jesse Yates 2012-09-13, 17:57
+
Andrew Purtell 2012-09-13, 18:31
+
n keywal 2012-09-13, 19:17
+
Andrew Purtell 2012-09-13, 20:20
+
n keywal 2012-09-13, 20:34
Copy link to this message
-
Re: read short circuit
Andrew Purtell 2012-09-13, 21:07
Yeah, on HBase-0.94-security

On Thu, Sep 13, 2012 at 1:34 PM, n keywal <[EMAIL PROTECTED]> wrote:
> We've got two security builds on 0.94: HBase-0.94-security and
> HBase-0.94-security-on-Hadoop-23
> We can change the setting on one of them?
>
> On Thu, Sep 13, 2012 at 10:20 PM, Andrew Purtell <[EMAIL PROTECTED]>wrote:
>
>> Hey N,
>>
>> It makes sense to have a 0.94 build with -Psecurity enabled and local
>> shortcut disabled. What do you think? For trunk builds your points sound
>> good, and I have no opinion.
>>
>> On Thursday, September 13, 2012, n keywal wrote:
>>
>> > If I sum-up :
>> >
>> > We make it the default:
>> > 1) Documented as such (what JD wrote already + a reference to HDFS-3907)
>> > 2) Set in the config we use for the unit tests.
>> >
>> > Andrew, you said that we should keep some test with it off ? May be we
>> > could do that in one of the existing build on the trunk, for example
>> > HBase-TRUNK-on-Hadoop-2.0.0? i.e.: We run the tests on trunk with
>> > short-circuit on; we run the test on hadoop-2 with short circuit off.
>> >
>> > I propose it this way because may be the tests will be faster with
>> > short-circuit on, and as well if it's the default production config it
>> > should be the one in the "main" test path.
>> >
>> > If that's ok, I can do test it on our test suite and do the jira for the
>> > integration.
>> >
>> >
>> >
>> >
>> > On Thu, Sep 13, 2012 at 8:31 PM, Andrew Purtell <[EMAIL PROTECTED]
>> <javascript:;>>
>> > wrote:
>> >
>> > > That's https://issues.apache.org/jira/browse/HBASE-4867
>> > >
>> > > On Thu, Sep 13, 2012 at 10:57 AM, Jesse Yates <[EMAIL PROTECTED]
>> <javascript:;>
>> > >
>> > > wrote:
>> > > > There was a jira a while back (can't remember the number) that I
>> think
>> > > came
>> > > > from the Facebook fellas to have multiple hbase-site.xmls, with a
>> > naming
>> > > > scheme such that you can do layering of configurations.
>> > > >
>> > > > What ever happened to that?
>> > > > -------------------
>> > > > Jesse Yates
>> > > > @jesse_yates
>> > > > jyates.github.com
>> > > >
>> > > >
>> > > > On Thu, Sep 13, 2012 at 10:44 AM, Andrew Purtell <
>> [EMAIL PROTECTED]<javascript:;>
>> > > >wrote:
>> > > >
>> > > >> Making it the default seems reasonable.
>> > > >>
>> > > >> For <= 0.94, the security profile has a separate hbase-site.xml file
>> > > >> under test resources where there it could be defaulted as off.
>>  Reason
>> > > >> is the local block sharing requires some shared group membership
>> > > >> between HBase and HDFS service accounts that might be objectionable
>> in
>> > > >> a secure setting. So we should have test coverage with it off.
>> > > >>
>> > > >> On Thu, Sep 13, 2012 at 9:42 AM, n keywal <[EMAIL PROTECTED]
>> <javascript:;>>
>> > wrote:
>> > > >> > It's all in the "pretty much" :-)
>> > > >> > Should we make it the default (i.e. recommending it in the doc,
>> > > emitting
>> > > >> a
>> > > >> > warning if it's not set) for all installation ?
>> > > >> > Or would it make sense to separate the store and wal cases,
>> meaning
>> > > that
>> > > >> we
>> > > >> > need a change in hdfs to make this possible?
>> > > >> >
>> > > >> >
>> > > >> > On Thu, Sep 13, 2012 at 6:30 PM, Jean-Daniel Cryans <
>> > > [EMAIL PROTECTED] <javascript:;>
>> > > >> >wrote:
>> > > >> >
>> > > >> >> On Thu, Sep 13, 2012 at 7:58 AM, n keywal <[EMAIL PROTECTED]
>> <javascript:;>>
>> > wrote:
>> > > >> >> > Hi,
>> > > >> >> >
>> > > >> >> > Is there any known counter indication for read short circuit?
>> > > >> >> > I had a look at JD's presentation from January. It was tested
>> vs.
>> > > >> HBase
>> > > >> >> as
>> > > >> >> > a whole; and still recommended for nearly all cases.
>> > > >> >>
>> > > >> >> Pretty much.
>> > > >> >>
>> > > >> >> >
>> > > >> >> > We could make it at least the default when reading HLog files
>> > (even
>> > > >> if it
>> > > >> >> > seems it's a 'per client" config, there is may be something to
>> > look
>> > > >> at as
>> > > >
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet
Hein (via Tom White)
+
n keywal 2012-09-14, 12:48
+
Jean-Daniel Cryans 2012-09-13, 16:51
+
Andrew Purtell 2012-09-13, 17:46
+
Stack 2012-09-13, 17:28
+
Jean-Daniel Cryans 2012-09-13, 18:01