Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> AggregateProtocol Help


+
Tom Wilcox 2011-12-22, 16:09
+
Ted Yu 2011-12-22, 17:03
+
Tom Wilcox 2011-12-23, 11:02
+
Ted Yu 2011-12-23, 15:04
Copy link to this message
-
Re: AggregateProtocol Help
Hi Tom,

The test code is not really the best guide for configuration.

To enable the AggregateProtocol on all of your tables, add this to the
hbase-site.xml for the servers in your cluster:

  <property>
    <name>hbase.coprocessor.user.region.classes</name>
    <value>org.apache.hadoop.hbase.coprocessor.AggregateImplementation</value>
  </property>

If you only want to use the aggregate functions on a specific table
(or tables), then you can enable that individually for the table from
the shell:

1) disable the table
hbase> disable 'EDRP7'

2) add the coprocessor
hbase> alter 'EDRP7', METHOD => 'table_att',
    'coprocessor'=>'|org.apache.hadoop.hbase.coprocessor.AggregateImplementation||'

(Note that the pipes in the value string are required)

3) re-enable the table
hbase> enable 'EDRP7'
Either way should work.  With the second approach you will see the
coprocessor listed when you describe the table from the shell, as Ted
mentioned.  With the first approach you will not, but it should be
loaded all the same.

--gh
On Fri, Dec 23, 2011 at 7:04 AM, Ted Yu <[EMAIL PROTECTED]> wrote:
> I don't know why you chose HBaseTestingUtility to create the table.
> I guess you followed test code example.
>
> At least you should pass the conf to this ctor:
>  public HBaseTestingUtility(Configuration conf) {
>
> If coprocessor was installed correctly, you should see something like(from
> HBASE-5070):
> coprocessor$1 =>
> '|org.apache.hadoop.hbase.constraint.ConstraintProcessor|1073741823|'
>
> Cheers
>
> On Fri, Dec 23, 2011 at 3:02 AM, Tom Wilcox <[EMAIL PROTECTED]> wrote:
>
>> Hi,
>>
>> I am not sure how we load the AggregateImplementation into the table. When
>> we are creating a table, we use the same functions as the test as follows...
>>
>> ...
>> >              conf.set(CoprocessorHost.REGION_COPROCESSOR_CONF_KEY,
>> >
>> > "org.apache.hadoop.hbase.coprocessor.AggregateImplementation");
>> >
>> >              // Utility.CreateHBaseTable(conf, otherArgs[1],
>> otherArgs[2],
>> > true);
>> >
>> >              HBaseTestingUtility util = new HBaseTestingUtility();
>> >              HTable table = util.createTable(EDRP_TABLE, EDRP_FAMILY);
>> >
>> >              AggregationClient aClient = new AggregationClient(conf);
>> ...
>>
>> Running DESCRIBE on a table produced shows the following output:
>>
>> hbase(main):002:0> describe 'EDRP7'
>> DESCRIPTION
>>                                            ENABLED
>>  {NAME => 'EDRP7', FAMILIES => [{NAME => 'advanceKWh', BLOOMFILTER =>
>> 'NONE', REPLICATION_SCOPE => '0', VERSIONS =>  true
>>  '3', COMPRESSION => 'NONE', MIN_VERSIONS => '0', TTL => '2147483647',
>> BLOCKSIZE => '65536', IN_MEMORY => 'false', B
>>  LOCKCACHE => 'true'}]}
>>
>> We are using the tip of 0.92 (cloned from the Git repo). See the version
>> string below:
>>
>> hbase(main):005:0> version
>> 0.92.0, r1208286, Thu Dec 15 13:16:03 GMT 2011
>>
>> We would really appreciate an example of how to create a table that is
>> enabled to handle Aggregation).
>>
>> Thanks
>>
>>
>> ________________________________________
>> From: Ted Yu [[EMAIL PROTECTED]]
>> Sent: 22 December 2011 17:03
>> To: [EMAIL PROTECTED]
>> Subject: Re: AggregateProtocol Help
>>
>> Have you loaded AggregateImplementation into your table ?
>> Can you show us the contents of the following command in hbase shell:
>> describe 'your-table'
>>
>> BTW are you using the tip of 0.92 ?
>> HBASE-4946 would be of help for dynamically loaded coprocessors which you
>> might use in the future.
>>
>> Cheers
>>
>> On Thu, Dec 22, 2011 at 8:09 AM, Tom Wilcox <[EMAIL PROTECTED]> wrote:
>>
>> > Hi,
>> >
>> > We are trying to use the aggregation functionality in HBase 0.92  and we
>> > have managed to get the test code working using the following command:
>> >
>> > java -classpath junit-4.10.jar:build/*:$HBASELIBS/*
>> > org.junit.runner.JUnitCore
>> > org.apache.hadoop.hbase.coprocessor.TestAggregateProtocol
>> >
>> > Closer inspection of this test class has revealed that it uses a mini DFS
+
Royston Sellman 2012-01-01, 17:35
+
Ted Yu 2012-01-01, 17:58
+
Royston Sellman 2012-01-01, 19:26
+
yuzhihong@... 2012-01-01, 19:53
+
Himanshu Vashishtha 2012-01-02, 01:18
+
Ted Yu 2012-01-02, 01:53
+
Gary Helmling 2012-01-02, 06:23
+
Royston Sellman 2012-01-03, 16:32
+
Ted Yu 2012-01-03, 17:09
+
Royston Sellman 2012-01-03, 17:48
+
Ted Yu 2012-01-03, 18:00
+
Himanshu Vashishtha 2012-01-03, 18:14
+
Royston Sellman 2012-01-03, 18:42
+
Ted Yu 2012-01-03, 18:58
+
Ted Yu 2012-01-03, 21:31
+
Royston Sellman 2012-01-04, 11:43
+
Ted Yu 2012-01-04, 15:01
+
Royston Sellman 2012-01-04, 18:57
+
Himanshu Vashishtha 2012-01-03, 17:11
+
Royston Sellman 2012-01-03, 17:54