Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - Coprocessor Aggregation supposed to be ~20x slower than Scans?


Copy link to this message
-
Re: Coprocessor Aggregation supposed to be ~20x slower than Scans?
anil gupta 2012-05-15, 23:58
Hi Ted,

I decompiled the hbase-0.92.0-cdh4b1.jar using JD-GUI and in
validateParameter method i don't find that condition.

Thanks,
Anil

On Tue, May 15, 2012 at 1:37 PM, Ted Yu <[EMAIL PROTECTED]> wrote:

> I checked the code in Apache HBase 0.92 and trunk. I see the following line
> in validateParameters():
>            !Bytes.equals(scan.getStopRow(), HConstants.EMPTY_END_ROW))) {
>
> Can you confirm that the bug is in cdh4b1 only ?
>
> Sorry for not doing the validation earlier.
>
> On Tue, May 15, 2012 at 12:09 PM, anil gupta <[EMAIL PROTECTED]>
> wrote:
>
> > Oh i c.. Now if i look closely at your gmail id then i can see your
> name. I
> > was totally confused.
> >
> > So, you want to force the user to specify stopRow if the filter is not
> > used? What if the user just wants to scan the table from startRow till
> the
> > end of table? In your solution user will have explicitly set the stopRow
> as
> > HConstants.EMPTY_END_ROW. Do we really want to force this?
> >
> > As per your solution the code would look like this:
> >      if(scan.hasFilter())
> >      {  if (scan == null || (Bytes.equals(scan.getStartRow(),
> > scan.getStopRow()) && !Bytes.equals(scan.getStartRow(),
> > HConstants.EMPTY_START_ROW)) || (Bytes.compareTo(scan.getStartRow(),
> > scan.getStopRow()) > 0 &&
> >             !Bytes.equals(scan.getStopRow(), HConstants.EMPTY_END_ROW)
> )) {
> >      throw new IOException(
> >          "Agg client Exception: Startrow should be smaller than
> Stoprow");
> >    } else if (scan.getFamilyMap().size() != 1) {
> >      throw new IOException("There must be only one family.");
> >    }
> >      }
> >      else
> >      {  if (scan == null || (Bytes.equals(scan.getStartRow(),
> > scan.getStopRow()) && !Bytes.equals(scan.getStartRow(),
> > HConstants.EMPTY_START_ROW)) || Bytes.compareTo(scan.getStartRow(),
> > scan.getStopRow()) > 0) {
> >          throw new IOException(
> >               "Agg client Exception: Startrow should be smaller than
> > Stoprow");
> >        } else if (scan.getFamilyMap().size() != 1) {
> >          throw new IOException("There must be only one family.");
> >        }
> >      }
> >
> > Let me know your thoughts.
> >
> > Thanks,
> > Anil
> >
> >
> > On Tue, May 15, 2012 at 11:46 AM, Ted Yu <[EMAIL PROTECTED]> wrote:
> >
> > > Anil:
> > > I am having trouble accessing JIRA.
> > >
> > > Ted Yu and Zhihong Yu are the same person :-)
> > >
> > > I think it would be good to remind user of aggregation client to narrow
> > > range of scan. That's why I proposed adding check of hasFilter().
> > >
> > > Cheers
> > >
> > > On Tue, May 15, 2012 at 10:47 AM, Ted Yu <[EMAIL PROTECTED]> wrote:
> > >
> > > > Take your time.
> > > > Once you complete your first submission, subsequent contributions
> would
> > > be
> > > > easier.
> > > >
> > > >
> > > > On Tue, May 15, 2012 at 10:34 AM, anil gupta <[EMAIL PROTECTED]
> > > >wrote:
> > > >
> > > >> Hi Ted,
> > > >>
> > > >> I created the jira:
> https://issues.apache.org/jira/browse/HBASE-5999for
> > > >> fixing this.
> > > >>
> > > >> Creating the patch might take me sometime(due to learning curve) as
> > this
> > > >> is
> > > >> the first time i would be creating a patch.
> > > >>
> > > >> Thanks,
> > > >> Anil Gupta
> > > >>
> > > >>
> > > >> On Mon, May 14, 2012 at 4:00 PM, Ted Yu <[EMAIL PROTECTED]>
> wrote:
> > > >>
> > > >> > I was aware of the following change.
> > > >> >
> > > >> > Can you log a JIRA and attach the patch to it ?
> > > >> >
> > > >> > Thanks for trying out and improving aggregation client.
> > > >> >
> > > >> > On Mon, May 14, 2012 at 3:31 PM, anil gupta <
> [EMAIL PROTECTED]>
> > > >> wrote:
> > > >> >
> > > >> > > Hi Ted,
> > > >> > >
> > > >> > > If we change the if statement condition in validateParameters
> > method
> > > >> in
> > > >> > > AggregationClient.java to:
> > > >> > > if (scan == null || (Bytes.equals(scan.getStartRow(),
> > > >> scan.getStopRow())
> > > >> > &&
> > > >> > > !Bytes.equals(scan.getStartRow(), HConstants.EMPTY_START_ROW))

Thanks & Regards,
Anil Gupta