Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # dev - read short circuit


+
n keywal 2012-09-13, 14:58
+
Jean-Daniel Cryans 2012-09-13, 16:30
+
n keywal 2012-09-13, 16:42
+
Andrew Purtell 2012-09-13, 17:44
+
Jesse Yates 2012-09-13, 17:57
+
Andrew Purtell 2012-09-13, 18:31
+
n keywal 2012-09-13, 19:17
Copy link to this message
-
Re: read short circuit
Andrew Purtell 2012-09-13, 20:20
Hey N,

It makes sense to have a 0.94 build with -Psecurity enabled and local
shortcut disabled. What do you think? For trunk builds your points sound
good, and I have no opinion.

On Thursday, September 13, 2012, n keywal wrote:

> If I sum-up :
>
> We make it the default:
> 1) Documented as such (what JD wrote already + a reference to HDFS-3907)
> 2) Set in the config we use for the unit tests.
>
> Andrew, you said that we should keep some test with it off ? May be we
> could do that in one of the existing build on the trunk, for example
> HBase-TRUNK-on-Hadoop-2.0.0? i.e.: We run the tests on trunk with
> short-circuit on; we run the test on hadoop-2 with short circuit off.
>
> I propose it this way because may be the tests will be faster with
> short-circuit on, and as well if it's the default production config it
> should be the one in the "main" test path.
>
> If that's ok, I can do test it on our test suite and do the jira for the
> integration.
>
>
>
>
> On Thu, Sep 13, 2012 at 8:31 PM, Andrew Purtell <[EMAIL PROTECTED]<javascript:;>>
> wrote:
>
> > That's https://issues.apache.org/jira/browse/HBASE-4867
> >
> > On Thu, Sep 13, 2012 at 10:57 AM, Jesse Yates <[EMAIL PROTECTED]<javascript:;>
> >
> > wrote:
> > > There was a jira a while back (can't remember the number) that I think
> > came
> > > from the Facebook fellas to have multiple hbase-site.xmls, with a
> naming
> > > scheme such that you can do layering of configurations.
> > >
> > > What ever happened to that?
> > > -------------------
> > > Jesse Yates
> > > @jesse_yates
> > > jyates.github.com
> > >
> > >
> > > On Thu, Sep 13, 2012 at 10:44 AM, Andrew Purtell <[EMAIL PROTECTED]<javascript:;>
> > >wrote:
> > >
> > >> Making it the default seems reasonable.
> > >>
> > >> For <= 0.94, the security profile has a separate hbase-site.xml file
> > >> under test resources where there it could be defaulted as off.  Reason
> > >> is the local block sharing requires some shared group membership
> > >> between HBase and HDFS service accounts that might be objectionable in
> > >> a secure setting. So we should have test coverage with it off.
> > >>
> > >> On Thu, Sep 13, 2012 at 9:42 AM, n keywal <[EMAIL PROTECTED]<javascript:;>>
> wrote:
> > >> > It's all in the "pretty much" :-)
> > >> > Should we make it the default (i.e. recommending it in the doc,
> > emitting
> > >> a
> > >> > warning if it's not set) for all installation ?
> > >> > Or would it make sense to separate the store and wal cases, meaning
> > that
> > >> we
> > >> > need a change in hdfs to make this possible?
> > >> >
> > >> >
> > >> > On Thu, Sep 13, 2012 at 6:30 PM, Jean-Daniel Cryans <
> > [EMAIL PROTECTED] <javascript:;>
> > >> >wrote:
> > >> >
> > >> >> On Thu, Sep 13, 2012 at 7:58 AM, n keywal <[EMAIL PROTECTED]<javascript:;>>
> wrote:
> > >> >> > Hi,
> > >> >> >
> > >> >> > Is there any known counter indication for read short circuit?
> > >> >> > I had a look at JD's presentation from January. It was tested vs.
> > >> HBase
> > >> >> as
> > >> >> > a whole; and still recommended for nearly all cases.
> > >> >>
> > >> >> Pretty much.
> > >> >>
> > >> >> >
> > >> >> > We could make it at least the default when reading HLog files
> (even
> > >> if it
> > >> >> > seems it's a 'per client" config, there is may be something to
> look
> > >> at as
> > >> >> > well)?
> > >> >>
> > >> >> You have to add 2 configs in hdfs-site.xml, which we can't do from
> > >> HBase.
> > >> >>
> > >> >> J-D
> > >> >>
> > >>
> > >>
> > >>
> > >> --
> > >> Best regards,
> > >>
> > >>    - Andy
> > >>
> > >> Problems worthy of attack prove their worth by hitting back. - Piet
> > >> Hein (via Tom White)
> > >>
> >
> >
> >
> > --
> > Best regards,
> >
> >    - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet
> > Hein (via Tom White)
> >
>
--
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
+
n keywal 2012-09-13, 20:34
+
Andrew Purtell 2012-09-13, 21:07
+
n keywal 2012-09-14, 12:48
+
Jean-Daniel Cryans 2012-09-13, 16:51
+
Andrew Purtell 2012-09-13, 17:46
+
Stack 2012-09-13, 17:28
+
Jean-Daniel Cryans 2012-09-13, 18:01