Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Coprocessor Aggregation supposed to be ~20x slower than Scans?


Copy link to this message
-
Re: Coprocessor Aggregation supposed to be ~20x slower than Scans?
I checked the code in Apache HBase 0.92 and trunk. I see the following line
in validateParameters():
            !Bytes.equals(scan.getStopRow(), HConstants.EMPTY_END_ROW))) {

Can you confirm that the bug is in cdh4b1 only ?

Sorry for not doing the validation earlier.

On Tue, May 15, 2012 at 12:09 PM, anil gupta <[EMAIL PROTECTED]> wrote:

> Oh i c.. Now if i look closely at your gmail id then i can see your name. I
> was totally confused.
>
> So, you want to force the user to specify stopRow if the filter is not
> used? What if the user just wants to scan the table from startRow till the
> end of table? In your solution user will have explicitly set the stopRow as
> HConstants.EMPTY_END_ROW. Do we really want to force this?
>
> As per your solution the code would look like this:
>      if(scan.hasFilter())
>      {  if (scan == null || (Bytes.equals(scan.getStartRow(),
> scan.getStopRow()) && !Bytes.equals(scan.getStartRow(),
> HConstants.EMPTY_START_ROW)) || (Bytes.compareTo(scan.getStartRow(),
> scan.getStopRow()) > 0 &&
>             !Bytes.equals(scan.getStopRow(), HConstants.EMPTY_END_ROW) )) {
>      throw new IOException(
>          "Agg client Exception: Startrow should be smaller than Stoprow");
>    } else if (scan.getFamilyMap().size() != 1) {
>      throw new IOException("There must be only one family.");
>    }
>      }
>      else
>      {  if (scan == null || (Bytes.equals(scan.getStartRow(),
> scan.getStopRow()) && !Bytes.equals(scan.getStartRow(),
> HConstants.EMPTY_START_ROW)) || Bytes.compareTo(scan.getStartRow(),
> scan.getStopRow()) > 0) {
>          throw new IOException(
>               "Agg client Exception: Startrow should be smaller than
> Stoprow");
>        } else if (scan.getFamilyMap().size() != 1) {
>          throw new IOException("There must be only one family.");
>        }
>      }
>
> Let me know your thoughts.
>
> Thanks,
> Anil
>
>
> On Tue, May 15, 2012 at 11:46 AM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
> > Anil:
> > I am having trouble accessing JIRA.
> >
> > Ted Yu and Zhihong Yu are the same person :-)
> >
> > I think it would be good to remind user of aggregation client to narrow
> > range of scan. That's why I proposed adding check of hasFilter().
> >
> > Cheers
> >
> > On Tue, May 15, 2012 at 10:47 AM, Ted Yu <[EMAIL PROTECTED]> wrote:
> >
> > > Take your time.
> > > Once you complete your first submission, subsequent contributions would
> > be
> > > easier.
> > >
> > >
> > > On Tue, May 15, 2012 at 10:34 AM, anil gupta <[EMAIL PROTECTED]
> > >wrote:
> > >
> > >> Hi Ted,
> > >>
> > >> I created the jira:https://issues.apache.org/jira/browse/HBASE-5999for
> > >> fixing this.
> > >>
> > >> Creating the patch might take me sometime(due to learning curve) as
> this
> > >> is
> > >> the first time i would be creating a patch.
> > >>
> > >> Thanks,
> > >> Anil Gupta
> > >>
> > >>
> > >> On Mon, May 14, 2012 at 4:00 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
> > >>
> > >> > I was aware of the following change.
> > >> >
> > >> > Can you log a JIRA and attach the patch to it ?
> > >> >
> > >> > Thanks for trying out and improving aggregation client.
> > >> >
> > >> > On Mon, May 14, 2012 at 3:31 PM, anil gupta <[EMAIL PROTECTED]>
> > >> wrote:
> > >> >
> > >> > > Hi Ted,
> > >> > >
> > >> > > If we change the if statement condition in validateParameters
> method
> > >> in
> > >> > > AggregationClient.java to:
> > >> > > if (scan == null || (Bytes.equals(scan.getStartRow(),
> > >> scan.getStopRow())
> > >> > &&
> > >> > > !Bytes.equals(scan.getStartRow(), HConstants.EMPTY_START_ROW)) ||
> > >> > > (Bytes.compareTo(scan.getStartRow(), scan.getStopRow()) > 0 &&
> > >> > > *!Bytes.equals(scan.getStopRow(),
> > >> > > HConstants.EMPTY_END_ROW)* ))
> > >> > >
> > >> > > Condition specified in the bold and Italic will handle the case
> when
> > >> the
> > >> > > stopRow is not specified. IMHO, it's not an error if we are not
> > >> > specifying
> > >> > > the stopRow. This is what is was looking for because in my case i