Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> coprocessor is timing out in 0.94


Copy link to this message
-
Re: coprocessor is timing out in 0.94
bq. the column we are applying filter against is in content family
I assume content family would always be returned in your query (after
filtering is done).

HBASE-5416 would help if you can specify metadata as essential column. This
would greatly reduce the amount of data loaded into memory.

Cheers

On Fri, Mar 29, 2013 at 6:49 AM, Agarwal, Saurabh
<[EMAIL PROTECTED]>wrote:

>  Thanks Marcos for reference. ****
>
> ** **
>
> We are using filter on only one column. And we further narrow down filter
> and results using ->****
>
> ** **
>
> scan.addColumn(family, qualifier);****
>
> ** **
>
> PageFilter pageFilter = *new* PageFilter(size);****
>
> mainFilterList.addFilter(pageFilter);****
>
> ** **
>
> we have two columns family – content ( most of the data) and metadata
> (only metadata). And the column we are applying filter against is in
> content family. In that case, will HBASE-5416 help?****
>
> ** **
>
> We are also using ROW bloomfilter. Will ROWCOlUMN bloomfilter help in our
> scenario? ****
>
> ** **
>
> ** **
>
> Saurabh. ****
>
> ** **
>
> ****
>
> *From:* Marcos Luis Ortiz Valmaseda [mailto:[EMAIL PROTECTED]]
> *Sent:* Thursday, March 28, 2013 10:36 PM
> *To:* [EMAIL PROTECTED]; Agarwal, Saurabh [CCC-OT_IT];
> [EMAIL PROTECTED]
>
> *Subject:* Re: coprocessor is timing out in 0.94****
>
> ** **
>
> Regards, Saurabh.****
>
> I see that you are using SingleColumnValueFilter. Look for these links:***
> *
>
> http://gbif.blogspot.com/2012/05/optimizing-hbase-mapreduce-scans-for.html
> ****
>
>
> http://mapredit.blogspot.com/2012/05/using-filters-in-hbase-to-match-two.html
> ****
>
> ** **
>
> Take a look later to this link, about the working to improve scans:****
>
> https://issues.apache.org/jira/browse/HBASE-5416****
>
> ** **
>
> ** **
>
> 2013/3/28 Agarwal, Saurabh <[EMAIL PROTECTED]>****
>
> Ted,
>
> Thanks for response.
>
> Here is the filter we are using -
> SingleColumnValueFilter(Bytes.toBytes(columnFamily),
> Bytes.toBytes(columnQualifier), CompareFilter.CompareOp.EQUAL, new
> RegexStringComparator("(?i)"+"keyword"));
>
> The thread dump at different points show that coprocessor is getting
> called. Also logs showed it keep processing. But the speed is much slower
> compare to 0.92.
>
> Regards,
> Saurabh.****
>
>
> -----Original Message-----
> From: Ted Yu [mailto:[EMAIL PROTECTED]]
> Sent: Thursday, March 28, 2013 6:57 PM
> To: [EMAIL PROTECTED]
> Subject: Re: coprocessor is timing out in 0.94
>
> bq. I checked thread dump
>
> If there was no exception in region server logs, thread dump of region
> server when your coprocessor was running would reveal where it got stuck.
>
> From your description below, looks like you can utilize HBASE-5416 Improve
> performance of scans with some kind of filters.
>
> bq. to apply the filter on one of the column
>
> Basically this column is the essential column.
>
> Cheers
>
> On Thu, Mar 28, 2013 at 3:22 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
> > bq. when I removed the filter, it ran fine in 0.94
> >
> > Can you disclose more information about your filter ?
> >
> > BTW 0.94.6 was just released which is fully compatible with 0.94.2
> >
> > Cheers
> >
> > On Thu, Mar 28, 2013 at 3:18 PM, Agarwal, Saurabh <
> > [EMAIL PROTECTED]> wrote:
> >
> >> Hi,
> >>
> >> We are in process of migrating from 0.92.1 to 0.94.2. A coprocessor
> >> was running fine in 0.92. After migrating to 0.94, the client is
> >> timing out (java.net.SocketTimeoutException).  We are using
> >> coprocessor to apply the filter on one of the column and return the
> >> columns that match with that filter criteria. I checked thread dump,
> >> region server, web UI, logs. There is no error or exception.  One
> >> thing I noticed that when I removed the filter, it ran fine in 0.94 as
> well.
> >>
> >> Please advise if there is any specific setting we need to make in 0.94.
> >>
> >> Thanks,
> >> Saurabh.
> >>
> >
> >****
>
>
>
> ****
>
> ** **
>
> -- ****
>
> Marcos Ortiz Valmaseda,