|
|
raghavendhra rahul 2012-03-20, 06:06
Hi, The coprocessor execution taking more time when i increase the number of regions in the table, why?
-
Re: Coprocessor execution
Stack 2012-03-20, 18:32
On Mon, Mar 19, 2012 at 11:06 PM, raghavendhra rahul <[EMAIL PROTECTED]> wrote: > Hi, > The coprocessor execution taking more time when i increase the number > of regions in the table, why? What is your CP doing? St.Ack
-
Re: Coprocessor execution
raghavendhra rahul 2012-03-21, 05:19
>What is your CP doing?
just rowcount only
> >
-
Re: Coprocessor execution
shixing 2012-03-21, 07:09
Use org.apache.hadoop.hbase.coprocessor.AggregateImplementation 's getRowNum() or you wrote the Coprocessor yourself? If you wrote it, paste it~ On Wed, Mar 21, 2012 at 1:19 PM, raghavendhra rahul < [EMAIL PROTECTED]> wrote:
> >What is your CP doing? > > just rowcount only > > > > > >
-- Best wishes! My Friend~
-
Re: Coprocessor execution
raghavendhra rahul 2012-03-21, 08:48
This is my coprocessor end point
public class RowcountEndpoint extends BaseEndpointCoprocessor implements RowcountProtocol {
@Override public long getRowcount() throws IOException { Scan scan = new Scan(); scan.setCaching(100); scan.setCacheBlocks(false); RegionCoprocessorEnvironment env (RegionCoprocessorEnvironment)getEnvironment(); InternalScanner scanner = env.getRegion().getScanner(scan); long result = 0; try{ List<KeyValue> val = new ArrayList<KeyValue>(); boolean done = false; do{ val.clear(); done = scanner.next(val); result+=1; }while(done); } finally{ scanner.close(); } return result; } } This is my client
public class RowClient { public Map<byte[], Long> getcount(String name) throws Throwable {
Configuration conf = HBaseConfiguration.create(); HTable table = new HTable(conf, name); Map<byte[], Long> res; try { res = table.coprocessorExec(RowcountProtocol.class, null, null, new Batch.Call<RowcountProtocol, Long>() { @Override public Long call(RowcountProtocol counter) throws IOException { return counter.getRowcount(); } }); } finally {
} return res; }
-
Re: Coprocessor execution
Himanshu Vashishtha 2012-03-21, 15:55
Hello,
Any info about specific numbers (number of regions vs response time, etc) will help.
Btw, for rowcount, you should use FirstKeyOnlyFilter. And in your code, you should add a callback to sum individual Region side results (though that is not related to response time, but with your rowcount final result). Thanks, Himanshu
On Wed, Mar 21, 2012 at 2:48 AM, raghavendhra rahul <[EMAIL PROTECTED]> wrote: > This is my coprocessor end point > > public class RowcountEndpoint extends BaseEndpointCoprocessor implements > RowcountProtocol { > > @Override > public long getRowcount() throws IOException { > Scan scan = new Scan(); > scan.setCaching(100); > scan.setCacheBlocks(false); > RegionCoprocessorEnvironment env > (RegionCoprocessorEnvironment)getEnvironment(); > InternalScanner scanner = env.getRegion().getScanner(scan); > long result = 0; > try{ > List<KeyValue> val = new ArrayList<KeyValue>(); > boolean done = false; > do{ > val.clear(); > done = scanner.next(val); > result+=1; > }while(done); > } > finally{ > scanner.close(); > } > return result; > } > } > > > This is my client > > public class RowClient { > public Map<byte[], Long> getcount(String name) throws Throwable { > > Configuration conf = HBaseConfiguration.create(); > HTable table = new HTable(conf, name); > Map<byte[], Long> res; > try { > res = table.coprocessorExec(RowcountProtocol.class, null, null, > new Batch.Call<RowcountProtocol, Long>() { > @Override > public Long call(RowcountProtocol counter) > throws IOException { > return counter.getRowcount(); > } > }); > } finally { > > } > return res; > }
|
|