|
Cyril Scetbon
2012-07-29, 21:49
yuzhihong@...
2012-07-29, 21:54
Himanshu Vashishtha
2012-07-29, 22:50
Cyril Scetbon
2012-07-30, 12:55
Cyril Scetbon
2012-07-30, 13:10
Ted Yu
2012-07-30, 15:34
Himanshu Vashishtha
2012-07-30, 15:56
Cyril Scetbon
2012-07-30, 16:43
Himanshu Vashishtha
2012-07-30, 17:30
Cyril Scetbon
2012-07-30, 21:15
|
-
Coprocessor POCCyril Scetbon 2012-07-29, 21:49
Hi,
I'm testing AggregationClient functions to check if we could use coprocessors for mathematical functions. The code I use is the following : package coreprocessor; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.hbase.HBaseConfiguration; import org.apache.hadoop.hbase.client.Scan; import org.apache.hadoop.hbase.client.coprocessor.AggregationClient; import org.apache.hadoop.hbase.util.Bytes; public class AggregationClientTest { private static final byte[] TABLE_NAME = Bytes.toBytes("ise"); private static final byte[] CF = Bytes.toBytes("core"); public static void main(String[] args) throws Throwable { Configuration configuration = HBaseConfiguration.create(); configuration.setLong("hbase.client.scanner.caching", 1000); AggregationClient aggregationClient = new AggregationClient( configuration); Scan scan = new Scan(); scan.addColumn(CF, Bytes.toBytes("value")); System.out.println("row count is " + aggregationClient.rowCount(TABLE_NAME, null, scan)); System.out.println("avg is " + aggregationClient.avg(TABLE_NAME, null, scan)); System.out.println("sum is " + aggregationClient.sum(TABLE_NAME, null, scan)); } } The only one working is the rowCount function. For others I get a NPE error ! I've checked that my table use only Long values for the column on which I work, and I've only one row in my table : ROW COLUMN+CELL id-cyr1 column=core:value, timestamp=1343596419845, value=\x00\x00\x00\x00\x00\x00\x00\x0A The only thing I can add is that my hbase server's version is 0.94.0 and that I use version 0.92.0 of the hbase jar any idea why it doesn't work ? thanks Cyril SCETBON
-
Re: Coprocessor POCyuzhihong@... 2012-07-29, 21:54
Can you use 0.94 for your client jar ?
Please show us the NullPointerException stack. Thanks On Jul 29, 2012, at 2:49 PM, Cyril Scetbon <[EMAIL PROTECTED]> wrote: > Hi, > > I'm testing AggregationClient functions to check if we could use coprocessors for mathematical functions. > > The code I use is the following : > > package coreprocessor; > > import org.apache.hadoop.conf.Configuration; > import org.apache.hadoop.hbase.HBaseConfiguration; > import org.apache.hadoop.hbase.client.Scan; > import org.apache.hadoop.hbase.client.coprocessor.AggregationClient; > import org.apache.hadoop.hbase.util.Bytes; > > public class AggregationClientTest { > > private static final byte[] TABLE_NAME = Bytes.toBytes("ise"); > private static final byte[] CF = Bytes.toBytes("core"); > > public static void main(String[] args) throws Throwable { > > Configuration configuration = HBaseConfiguration.create(); > > configuration.setLong("hbase.client.scanner.caching", 1000); > AggregationClient aggregationClient = new AggregationClient( > configuration); > Scan scan = new Scan(); > scan.addColumn(CF, Bytes.toBytes("value")); > System.out.println("row count is " + aggregationClient.rowCount(TABLE_NAME, null, scan)); > System.out.println("avg is " + aggregationClient.avg(TABLE_NAME, null, scan)); > System.out.println("sum is " + aggregationClient.sum(TABLE_NAME, null, scan)); > } > } > > The only one working is the rowCount function. For others I get a NPE error ! > I've checked that my table use only Long values for the column on which I work, and I've only one row in my table : > > ROW COLUMN+CELL > id-cyr1 column=core:value, timestamp=1343596419845, value=\x00\x00\x00\x00\x00\x00\x00\x0A > > The only thing I can add is that my hbase server's version is 0.94.0 and that I use version 0.92.0 of the hbase jar > > any idea why it doesn't work ? > > thanks > Cyril SCETBON >
-
Re: Coprocessor POCHimanshu Vashishtha 2012-07-29, 22:50
And also, what are your cell values look like?
Himanshu On Sun, Jul 29, 2012 at 3:54 PM, <[EMAIL PROTECTED]> wrote: > Can you use 0.94 for your client jar ? > > Please show us the NullPointerException stack. > > Thanks > > > > On Jul 29, 2012, at 2:49 PM, Cyril Scetbon <[EMAIL PROTECTED]> wrote: > >> Hi, >> >> I'm testing AggregationClient functions to check if we could use coprocessors for mathematical functions. >> >> The code I use is the following : >> >> package coreprocessor; >> >> import org.apache.hadoop.conf.Configuration; >> import org.apache.hadoop.hbase.HBaseConfiguration; >> import org.apache.hadoop.hbase.client.Scan; >> import org.apache.hadoop.hbase.client.coprocessor.AggregationClient; >> import org.apache.hadoop.hbase.util.Bytes; >> >> public class AggregationClientTest { >> >> private static final byte[] TABLE_NAME = Bytes.toBytes("ise"); >> private static final byte[] CF = Bytes.toBytes("core"); >> >> public static void main(String[] args) throws Throwable { >> >> Configuration configuration = HBaseConfiguration.create(); >> >> configuration.setLong("hbase.client.scanner.caching", 1000); >> AggregationClient aggregationClient = new AggregationClient( >> configuration); >> Scan scan = new Scan(); >> scan.addColumn(CF, Bytes.toBytes("value")); >> System.out.println("row count is " + aggregationClient.rowCount(TABLE_NAME, null, scan)); >> System.out.println("avg is " + aggregationClient.avg(TABLE_NAME, null, scan)); >> System.out.println("sum is " + aggregationClient.sum(TABLE_NAME, null, scan)); >> } >> } >> >> The only one working is the rowCount function. For others I get a NPE error ! >> I've checked that my table use only Long values for the column on which I work, and I've only one row in my table : >> >> ROW COLUMN+CELL >> id-cyr1 column=core:value, timestamp=1343596419845, value=\x00\x00\x00\x00\x00\x00\x00\x0A >> >> The only thing I can add is that my hbase server's version is 0.94.0 and that I use version 0.92.0 of the hbase jar >> >> any idea why it doesn't work ? >> >> thanks >> Cyril SCETBON >>
-
Re: Coprocessor POCCyril Scetbon 2012-07-30, 12:55
I've given the values returned by scan 'table' command in hbase shell in my first email.
Regards Cyril SCETBON On Jul 30, 2012, at 12:50 AM, Himanshu Vashishtha <[EMAIL PROTECTED]> wrote: > And also, what are your cell values look like? > > Himanshu > > On Sun, Jul 29, 2012 at 3:54 PM, <[EMAIL PROTECTED]> wrote: >> Can you use 0.94 for your client jar ? >> >> Please show us the NullPointerException stack. >> >> Thanks >> >> >> >> On Jul 29, 2012, at 2:49 PM, Cyril Scetbon <[EMAIL PROTECTED]> wrote: >> >>> Hi, >>> >>> I'm testing AggregationClient functions to check if we could use coprocessors for mathematical functions. >>> >>> The code I use is the following : >>> >>> package coreprocessor; >>> >>> import org.apache.hadoop.conf.Configuration; >>> import org.apache.hadoop.hbase.HBaseConfiguration; >>> import org.apache.hadoop.hbase.client.Scan; >>> import org.apache.hadoop.hbase.client.coprocessor.AggregationClient; >>> import org.apache.hadoop.hbase.util.Bytes; >>> >>> public class AggregationClientTest { >>> >>> private static final byte[] TABLE_NAME = Bytes.toBytes("ise"); >>> private static final byte[] CF = Bytes.toBytes("core"); >>> >>> public static void main(String[] args) throws Throwable { >>> >>> Configuration configuration = HBaseConfiguration.create(); >>> >>> configuration.setLong("hbase.client.scanner.caching", 1000); >>> AggregationClient aggregationClient = new AggregationClient( >>> configuration); >>> Scan scan = new Scan(); >>> scan.addColumn(CF, Bytes.toBytes("value")); >>> System.out.println("row count is " + aggregationClient.rowCount(TABLE_NAME, null, scan)); >>> System.out.println("avg is " + aggregationClient.avg(TABLE_NAME, null, scan)); >>> System.out.println("sum is " + aggregationClient.sum(TABLE_NAME, null, scan)); >>> } >>> } >>> >>> The only one working is the rowCount function. For others I get a NPE error ! >>> I've checked that my table use only Long values for the column on which I work, and I've only one row in my table : >>> >>> ROW COLUMN+CELL >>> id-cyr1 column=core:value, timestamp=1343596419845, value=\x00\x00\x00\x00\x00\x00\x00\x0A >>> >>> The only thing I can add is that my hbase server's version is 0.94.0 and that I use version 0.92.0 of the hbase jar >>> >>> any idea why it doesn't work ? >>> >>> thanks >>> Cyril SCETBON >>>
-
Re: Coprocessor POCCyril Scetbon 2012-07-30, 13:10
Here is the stack : (with hbase-0.94.0.jar)
row count is 1 12/07/30 15:08:53 WARN client.HConnectionManager$HConnectionImplementation: Error executing for row java.util.concurrent.ExecutionException: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=10, exceptions: Mon Jul 30 15:08:14 CEST 2012, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@341049d3, java.io.IOException: java.io.IOException: java.lang.NullPointerException at org.apache.hadoop.hbase.coprocessor.AggregateImplementation.getAvg(AggregateImplementation.java:189) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.regionserver.HRegion.exec(HRegion.java:4770) at org.apache.hadoop.hbase.regionserver.HRegionServer.execCoprocessor(HRegionServer.java:3457) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364) at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1376) Regards Cyril SCETBON On Jul 29, 2012, at 11:54 PM, [EMAIL PROTECTED] wrote: > Can you use 0.94 for your client jar ? > > Please show us the NullPointerException stack. > > Thanks > > > > On Jul 29, 2012, at 2:49 PM, Cyril Scetbon <[EMAIL PROTECTED]> wrote: > >> Hi, >> >> I'm testing AggregationClient functions to check if we could use coprocessors for mathematical functions. >> >> The code I use is the following : >> >> package coreprocessor; >> >> import org.apache.hadoop.conf.Configuration; >> import org.apache.hadoop.hbase.HBaseConfiguration; >> import org.apache.hadoop.hbase.client.Scan; >> import org.apache.hadoop.hbase.client.coprocessor.AggregationClient; >> import org.apache.hadoop.hbase.util.Bytes; >> >> public class AggregationClientTest { >> >> private static final byte[] TABLE_NAME = Bytes.toBytes("ise"); >> private static final byte[] CF = Bytes.toBytes("core"); >> >> public static void main(String[] args) throws Throwable { >> >> Configuration configuration = HBaseConfiguration.create(); >> >> configuration.setLong("hbase.client.scanner.caching", 1000); >> AggregationClient aggregationClient = new AggregationClient( >> configuration); >> Scan scan = new Scan(); >> scan.addColumn(CF, Bytes.toBytes("value")); >> System.out.println("row count is " + aggregationClient.rowCount(TABLE_NAME, null, scan)); >> System.out.println("avg is " + aggregationClient.avg(TABLE_NAME, null, scan)); >> System.out.println("sum is " + aggregationClient.sum(TABLE_NAME, null, scan)); >> } >> } >> >> The only one working is the rowCount function. For others I get a NPE error ! >> I've checked that my table use only Long values for the column on which I work, and I've only one row in my table : >> >> ROW COLUMN+CELL >> id-cyr1 column=core:value, timestamp=1343596419845, value=\x00\x00\x00\x00\x00\x00\x00\x0A >> >> The only thing I can add is that my hbase server's version is 0.94.0 and that I use version 0.92.0 of the hbase jar >> >> any idea why it doesn't work ? >> >> thanks >> Cyril SCETBON >>
-
Re: Coprocessor POCTed Yu 2012-07-30, 15:34
Did your client include the following fix ?
HBASE-5821 Incorrect handling of null value in Coprocessor aggregation function min() (Maryann Xue) >From the stack trace you provided, looks like the NPE came from this line: sumVal = ci.add(sumVal, ci.castToReturnType(ci.getValue(colFamily, qualifier, kv))); ci.getValue() might have returned null. Please update your client to 0.94.1 RC and see if the NPE occurs. Thanks On Mon, Jul 30, 2012 at 6:10 AM, Cyril Scetbon <[EMAIL PROTECTED]>wrote: > Here is the stack : (with hbase-0.94.0.jar) > > row count is 1 > 12/07/30 15:08:53 WARN > client.HConnectionManager$HConnectionImplementation: Error executing for row > java.util.concurrent.ExecutionException: > org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after > attempts=10, exceptions: > Mon Jul 30 15:08:14 CEST 2012, > org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@341049d3, > java.io.IOException: java.io.IOException: java.lang.NullPointerException > at > org.apache.hadoop.hbase.coprocessor.AggregateImplementation.getAvg(AggregateImplementation.java:189) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.apache.hadoop.hbase.regionserver.HRegion.exec(HRegion.java:4770) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.execCoprocessor(HRegionServer.java:3457) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364) > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1376) > > Regards > Cyril SCETBON > > On Jul 29, 2012, at 11:54 PM, [EMAIL PROTECTED] wrote: > > > Can you use 0.94 for your client jar ? > > > > Please show us the NullPointerException stack. > > > > Thanks > > > > > > > > On Jul 29, 2012, at 2:49 PM, Cyril Scetbon <[EMAIL PROTECTED]> > wrote: > > > >> Hi, > >> > >> I'm testing AggregationClient functions to check if we could use > coprocessors for mathematical functions. > >> > >> The code I use is the following : > >> > >> package coreprocessor; > >> > >> import org.apache.hadoop.conf.Configuration; > >> import org.apache.hadoop.hbase.HBaseConfiguration; > >> import org.apache.hadoop.hbase.client.Scan; > >> import org.apache.hadoop.hbase.client.coprocessor.AggregationClient; > >> import org.apache.hadoop.hbase.util.Bytes; > >> > >> public class AggregationClientTest { > >> > >> private static final byte[] TABLE_NAME = Bytes.toBytes("ise"); > >> private static final byte[] CF = Bytes.toBytes("core"); > >> > >> public static void main(String[] args) throws Throwable { > >> > >> Configuration configuration = HBaseConfiguration.create(); > >> > >> configuration.setLong("hbase.client.scanner.caching", 1000); > >> AggregationClient aggregationClient = new AggregationClient( > >> configuration); > >> Scan scan = new Scan(); > >> scan.addColumn(CF, Bytes.toBytes("value")); > >> System.out.println("row count is " + > aggregationClient.rowCount(TABLE_NAME, null, scan)); > >> System.out.println("avg is " + aggregationClient.avg(TABLE_NAME, > null, scan)); > >> System.out.println("sum is " + aggregationClient.sum(TABLE_NAME, > null, scan)); > >> } > >> } > >> > >> The only one working is the rowCount function. For others I get a NPE > error ! > >> I've checked that my table use only Long values for the column on which
-
Re: Coprocessor POCHimanshu Vashishtha 2012-07-30, 15:56
On Mon, Jul 30, 2012 at 6:55 AM, Cyril Scetbon <[EMAIL PROTECTED]> wrote:
> I've given the values returned by scan 'table' command in hbase shell in my first email. Somehow I missed the scan result in your first email. So, can you pass a LongColumnInterpreter instance instead of null? See TestAggregateProtocol methods for usage. Thanks Himanshu > > Regards > Cyril SCETBON > > On Jul 30, 2012, at 12:50 AM, Himanshu Vashishtha <[EMAIL PROTECTED]> wrote: > >> And also, what are your cell values look like? >> >> Himanshu >> >> On Sun, Jul 29, 2012 at 3:54 PM, <[EMAIL PROTECTED]> wrote: >>> Can you use 0.94 for your client jar ? >>> >>> Please show us the NullPointerException stack. >>> >>> Thanks >>> >>> >>> >>> On Jul 29, 2012, at 2:49 PM, Cyril Scetbon <[EMAIL PROTECTED]> wrote: >>> >>>> Hi, >>>> >>>> I'm testing AggregationClient functions to check if we could use coprocessors for mathematical functions. >>>> >>>> The code I use is the following : >>>> >>>> package coreprocessor; >>>> >>>> import org.apache.hadoop.conf.Configuration; >>>> import org.apache.hadoop.hbase.HBaseConfiguration; >>>> import org.apache.hadoop.hbase.client.Scan; >>>> import org.apache.hadoop.hbase.client.coprocessor.AggregationClient; >>>> import org.apache.hadoop.hbase.util.Bytes; >>>> >>>> public class AggregationClientTest { >>>> >>>> private static final byte[] TABLE_NAME = Bytes.toBytes("ise"); >>>> private static final byte[] CF = Bytes.toBytes("core"); >>>> >>>> public static void main(String[] args) throws Throwable { >>>> >>>> Configuration configuration = HBaseConfiguration.create(); >>>> >>>> configuration.setLong("hbase.client.scanner.caching", 1000); >>>> AggregationClient aggregationClient = new AggregationClient( >>>> configuration); >>>> Scan scan = new Scan(); >>>> scan.addColumn(CF, Bytes.toBytes("value")); >>>> System.out.println("row count is " + aggregationClient.rowCount(TABLE_NAME, null, scan)); >>>> System.out.println("avg is " + aggregationClient.avg(TABLE_NAME, null, scan)); >>>> System.out.println("sum is " + aggregationClient.sum(TABLE_NAME, null, scan)); >>>> } >>>> } >>>> >>>> The only one working is the rowCount function. For others I get a NPE error ! >>>> I've checked that my table use only Long values for the column on which I work, and I've only one row in my table : >>>> >>>> ROW COLUMN+CELL >>>> id-cyr1 column=core:value, timestamp=1343596419845, value=\x00\x00\x00\x00\x00\x00\x00\x0A >>>> >>>> The only thing I can add is that my hbase server's version is 0.94.0 and that I use version 0.92.0 of the hbase jar >>>> >>>> any idea why it doesn't work ? >>>> >>>> thanks >>>> Cyril SCETBON >>>> >
-
Re: Coprocessor POCCyril Scetbon 2012-07-30, 16:43
Thanks, it's really better !
I've read that by default it supports only Long values, that's why I was using a null ColumnInterpreter. Regards. Cyril SCETBON On Jul 30, 2012, at 5:56 PM, Himanshu Vashishtha <[EMAIL PROTECTED]> wrote: > On Mon, Jul 30, 2012 at 6:55 AM, Cyril Scetbon <[EMAIL PROTECTED]> wrote: > >> I've given the values returned by scan 'table' command in hbase shell in my first email. > Somehow I missed the scan result in your first email. So, can you pass > a LongColumnInterpreter instance instead of null? > See TestAggregateProtocol methods for usage. > > Thanks > Himanshu > >> >> Regards >> Cyril SCETBON >> >> On Jul 30, 2012, at 12:50 AM, Himanshu Vashishtha <[EMAIL PROTECTED]> wrote: >> >>> And also, what are your cell values look like? >>> >>> Himanshu >>> >>> On Sun, Jul 29, 2012 at 3:54 PM, <[EMAIL PROTECTED]> wrote: >>>> Can you use 0.94 for your client jar ? >>>> >>>> Please show us the NullPointerException stack. >>>> >>>> Thanks >>>> >>>> >>>> >>>> On Jul 29, 2012, at 2:49 PM, Cyril Scetbon <[EMAIL PROTECTED]> wrote: >>>> >>>>> Hi, >>>>> >>>>> I'm testing AggregationClient functions to check if we could use coprocessors for mathematical functions. >>>>> >>>>> The code I use is the following : >>>>> >>>>> package coreprocessor; >>>>> >>>>> import org.apache.hadoop.conf.Configuration; >>>>> import org.apache.hadoop.hbase.HBaseConfiguration; >>>>> import org.apache.hadoop.hbase.client.Scan; >>>>> import org.apache.hadoop.hbase.client.coprocessor.AggregationClient; >>>>> import org.apache.hadoop.hbase.util.Bytes; >>>>> >>>>> public class AggregationClientTest { >>>>> >>>>> private static final byte[] TABLE_NAME = Bytes.toBytes("ise"); >>>>> private static final byte[] CF = Bytes.toBytes("core"); >>>>> >>>>> public static void main(String[] args) throws Throwable { >>>>> >>>>> Configuration configuration = HBaseConfiguration.create(); >>>>> >>>>> configuration.setLong("hbase.client.scanner.caching", 1000); >>>>> AggregationClient aggregationClient = new AggregationClient( >>>>> configuration); >>>>> Scan scan = new Scan(); >>>>> scan.addColumn(CF, Bytes.toBytes("value")); >>>>> System.out.println("row count is " + aggregationClient.rowCount(TABLE_NAME, null, scan)); >>>>> System.out.println("avg is " + aggregationClient.avg(TABLE_NAME, null, scan)); >>>>> System.out.println("sum is " + aggregationClient.sum(TABLE_NAME, null, scan)); >>>>> } >>>>> } >>>>> >>>>> The only one working is the rowCount function. For others I get a NPE error ! >>>>> I've checked that my table use only Long values for the column on which I work, and I've only one row in my table : >>>>> >>>>> ROW COLUMN+CELL >>>>> id-cyr1 column=core:value, timestamp=1343596419845, value=\x00\x00\x00\x00\x00\x00\x00\x0A >>>>> >>>>> The only thing I can add is that my hbase server's version is 0.94.0 and that I use version 0.92.0 of the hbase jar >>>>> >>>>> any idea why it doesn't work ? >>>>> >>>>> thanks >>>>> Cyril SCETBON >>>>> >>
-
Re: Coprocessor POCHimanshu Vashishtha 2012-07-30, 17:30
We should fix the reference then. Where did you read it?
On Mon, Jul 30, 2012 at 10:43 AM, Cyril Scetbon <[EMAIL PROTECTED]> wrote: > Thanks, it's really better ! > > I've read that by default it supports only Long values, that's why I was using a null ColumnInterpreter. > > Regards. > Cyril SCETBON > > On Jul 30, 2012, at 5:56 PM, Himanshu Vashishtha <[EMAIL PROTECTED]> wrote: > >> On Mon, Jul 30, 2012 at 6:55 AM, Cyril Scetbon <[EMAIL PROTECTED]> wrote: >> >>> I've given the values returned by scan 'table' command in hbase shell in my first email. >> Somehow I missed the scan result in your first email. So, can you pass >> a LongColumnInterpreter instance instead of null? >> See TestAggregateProtocol methods for usage. >> >> Thanks >> Himanshu >> >>> >>> Regards >>> Cyril SCETBON >>> >>> On Jul 30, 2012, at 12:50 AM, Himanshu Vashishtha <[EMAIL PROTECTED]> wrote: >>> >>>> And also, what are your cell values look like? >>>> >>>> Himanshu >>>> >>>> On Sun, Jul 29, 2012 at 3:54 PM, <[EMAIL PROTECTED]> wrote: >>>>> Can you use 0.94 for your client jar ? >>>>> >>>>> Please show us the NullPointerException stack. >>>>> >>>>> Thanks >>>>> >>>>> >>>>> >>>>> On Jul 29, 2012, at 2:49 PM, Cyril Scetbon <[EMAIL PROTECTED]> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> I'm testing AggregationClient functions to check if we could use coprocessors for mathematical functions. >>>>>> >>>>>> The code I use is the following : >>>>>> >>>>>> package coreprocessor; >>>>>> >>>>>> import org.apache.hadoop.conf.Configuration; >>>>>> import org.apache.hadoop.hbase.HBaseConfiguration; >>>>>> import org.apache.hadoop.hbase.client.Scan; >>>>>> import org.apache.hadoop.hbase.client.coprocessor.AggregationClient; >>>>>> import org.apache.hadoop.hbase.util.Bytes; >>>>>> >>>>>> public class AggregationClientTest { >>>>>> >>>>>> private static final byte[] TABLE_NAME = Bytes.toBytes("ise"); >>>>>> private static final byte[] CF = Bytes.toBytes("core"); >>>>>> >>>>>> public static void main(String[] args) throws Throwable { >>>>>> >>>>>> Configuration configuration = HBaseConfiguration.create(); >>>>>> >>>>>> configuration.setLong("hbase.client.scanner.caching", 1000); >>>>>> AggregationClient aggregationClient = new AggregationClient( >>>>>> configuration); >>>>>> Scan scan = new Scan(); >>>>>> scan.addColumn(CF, Bytes.toBytes("value")); >>>>>> System.out.println("row count is " + aggregationClient.rowCount(TABLE_NAME, null, scan)); >>>>>> System.out.println("avg is " + aggregationClient.avg(TABLE_NAME, null, scan)); >>>>>> System.out.println("sum is " + aggregationClient.sum(TABLE_NAME, null, scan)); >>>>>> } >>>>>> } >>>>>> >>>>>> The only one working is the rowCount function. For others I get a NPE error ! >>>>>> I've checked that my table use only Long values for the column on which I work, and I've only one row in my table : >>>>>> >>>>>> ROW COLUMN+CELL >>>>>> id-cyr1 column=core:value, timestamp=1343596419845, value=\x00\x00\x00\x00\x00\x00\x00\x0A >>>>>> >>>>>> The only thing I can add is that my hbase server's version is 0.94.0 and that I use version 0.92.0 of the hbase jar >>>>>> >>>>>> any idea why it doesn't work ? >>>>>> >>>>>> thanks >>>>>> Cyril SCETBON >>>>>> >>> >
-
Re: Coprocessor POCCyril Scetbon 2012-07-30, 21:15
unfortunately I can't remember/find it :( and I see in AggregationClient's javadoc that :
"Column family can't be null" , so I suppose I should have read it at first ! Thanks again Cyril SCETBON On Jul 30, 2012, at 7:30 PM, Himanshu Vashishtha <[EMAIL PROTECTED]> wrote: > We should fix the reference then. Where did you read it? > > On Mon, Jul 30, 2012 at 10:43 AM, Cyril Scetbon <[EMAIL PROTECTED]> wrote: >> Thanks, it's really better ! >> >> I've read that by default it supports only Long values, that's why I was using a null ColumnInterpreter. >> >> Regards. >> Cyril SCETBON >> >> On Jul 30, 2012, at 5:56 PM, Himanshu Vashishtha <[EMAIL PROTECTED]> wrote: >> >>> On Mon, Jul 30, 2012 at 6:55 AM, Cyril Scetbon <[EMAIL PROTECTED]> wrote: >>> >>>> I've given the values returned by scan 'table' command in hbase shell in my first email. >>> Somehow I missed the scan result in your first email. So, can you pass >>> a LongColumnInterpreter instance instead of null? >>> See TestAggregateProtocol methods for usage. >>> >>> Thanks >>> Himanshu >>> >>>> >>>> Regards >>>> Cyril SCETBON >>>> >>>> On Jul 30, 2012, at 12:50 AM, Himanshu Vashishtha <[EMAIL PROTECTED]> wrote: >>>> >>>>> And also, what are your cell values look like? >>>>> >>>>> Himanshu >>>>> >>>>> On Sun, Jul 29, 2012 at 3:54 PM, <[EMAIL PROTECTED]> wrote: >>>>>> Can you use 0.94 for your client jar ? >>>>>> >>>>>> Please show us the NullPointerException stack. >>>>>> >>>>>> Thanks >>>>>> >>>>>> >>>>>> >>>>>> On Jul 29, 2012, at 2:49 PM, Cyril Scetbon <[EMAIL PROTECTED]> wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> I'm testing AggregationClient functions to check if we could use coprocessors for mathematical functions. >>>>>>> >>>>>>> The code I use is the following : >>>>>>> >>>>>>> package coreprocessor; >>>>>>> >>>>>>> import org.apache.hadoop.conf.Configuration; >>>>>>> import org.apache.hadoop.hbase.HBaseConfiguration; >>>>>>> import org.apache.hadoop.hbase.client.Scan; >>>>>>> import org.apache.hadoop.hbase.client.coprocessor.AggregationClient; >>>>>>> import org.apache.hadoop.hbase.util.Bytes; >>>>>>> >>>>>>> public class AggregationClientTest { >>>>>>> >>>>>>> private static final byte[] TABLE_NAME = Bytes.toBytes("ise"); >>>>>>> private static final byte[] CF = Bytes.toBytes("core"); >>>>>>> >>>>>>> public static void main(String[] args) throws Throwable { >>>>>>> >>>>>>> Configuration configuration = HBaseConfiguration.create(); >>>>>>> >>>>>>> configuration.setLong("hbase.client.scanner.caching", 1000); >>>>>>> AggregationClient aggregationClient = new AggregationClient( >>>>>>> configuration); >>>>>>> Scan scan = new Scan(); >>>>>>> scan.addColumn(CF, Bytes.toBytes("value")); >>>>>>> System.out.println("row count is " + aggregationClient.rowCount(TABLE_NAME, null, scan)); >>>>>>> System.out.println("avg is " + aggregationClient.avg(TABLE_NAME, null, scan)); >>>>>>> System.out.println("sum is " + aggregationClient.sum(TABLE_NAME, null, scan)); >>>>>>> } >>>>>>> } >>>>>>> >>>>>>> The only one working is the rowCount function. For others I get a NPE error ! >>>>>>> I've checked that my table use only Long values for the column on which I work, and I've only one row in my table : >>>>>>> >>>>>>> ROW COLUMN+CELL >>>>>>> id-cyr1 column=core:value, timestamp=1343596419845, value=\x00\x00\x00\x00\x00\x00\x00\x0A >>>>>>> >>>>>>> The only thing I can add is that my hbase server's version is 0.94.0 and that I use version 0.92.0 of the hbase jar >>>>>>> >>>>>>> any idea why it doesn't work ? >>>>>>> >>>>>>> thanks >>>>>>> Cyril SCETBON >>>>>>> >>>> >> |