Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # general >> LimitedPrivate and HBase

Copy link to this message
Re: LimitedPrivate and HBase
> I can tell you feel I'm picking on HBase, especially in light of my
> flat out rejection of the "we want to mmap() blocks" case.

I for one understand the objection there.

Although it does negatively impact the work of a recent promising new contributor. As a project, HBase suffers for it. Of course that is no concern of HDFS.

On the other hand I do believe Todd has a point. MapReduce is perhaps the only constituency that HDFS really cares about. Any reasonable person would come to that conclusion after surveying submitted JIRAs and their resolution times (or not). Historically with HDFS the local itch, the concern of the big MapReduce shops, gets the scratch and others are of not much concern. Therefore there is unfortunate business that lingers today -- Facebook, StumbleUpon, Trend Micro, and others have effectively forked HDFS (0.20) in house for use with HBase, and nobody I know is seriously considering using HDFS 0.22 or TRUNK due to a lack of evidence that anyone with a stake in it is running it in production at scale. Past discussion to mend the breach with an HBase-friendly release of HDFS 0.20 ended with what I would describe as an inflexible and legalistic air.

Best regards,

    - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
--- On Mon, 6/6/11, Allen Wittenauer <[EMAIL PROTECTED]> wrote:

> From: Allen Wittenauer <[EMAIL PROTECTED]>
> Subject: Re: LimitedPrivate and HBase
> Date: Monday, June 6, 2011, 6:18 PM
> On Jun 6, 2011, at 6:08 PM, Todd Lipcon wrote:
> >
> >>
> >>       Let's face it: this
> happened because it was HBase.  If it was almost
> >> anyone else, it would have sat there.... and
> *that's* the point where I'm
> >> mainly concerned.
> >
> >
> > If you want to feel better, take a look at HDFS-941,
> HDFS-347, and HDFS-918
> > - these are patches that HBase has been asking for for
> nearly 2 years in
> > some cases and haven't gone in. Satisfied?
>     These cases don't appear to be about
> re-classification of an API from private to
> semi-public.  So no, I'm not.  None of these
> appear to answer the base set of question:
>     - What is the real criteria for changing
> an API from private to limited?
>     - How "closely related" does a project
> need to be to get this privilege?
>     (Yes, I've read the classification
> docs.  That's too vague.)
>     I can tell you feel I'm picking on
> HBase, especially in light of my flat out rejection of the
> "we want to mmap() blocks" case.  But if this
> reclassification had been with anything else outside of the
> Hadoop project, I would have asked the same thing.  It
> raises important questions that we as a project need to
> answer.