|
|
-
how to use API to statistic how many message has been store in the table in hbase
tgh 2012-12-27, 02:21
Hi I try to use hbaseAPI to store data, and I want to get the number of the message stored in the table in HBase, how to use API to statistic how many message has been store in the table in hbase What API can do this, Could you help me Thank you --------------------- Tian Guanhua
-
RE: how to use API to statistic how many message has been store in the table in hbase
Anoop Sam John 2012-12-27, 05:31
So you want to know the no# of rows in a table? Have a look at AggregationClient#rowCount()
-Anoop- ________________________________________ From: tgh [[EMAIL PROTECTED]] Sent: Thursday, December 27, 2012 7:51 AM To: [EMAIL PROTECTED] Subject: how to use API to statistic how many message has been store in the table in hbase
Hi I try to use hbaseAPI to store data, and I want to get the number of the message stored in the table in HBase, how to use API to statistic how many message has been store in the table in hbase What API can do this, Could you help me Thank you --------------------- Tian Guanhua
-
答复: how to use API to statistic how many message has been store in the table in hbase
tgh 2012-12-27, 07:16
Thank you for your reply And I AggregationClient#rowCount()
But I am not familiar with these API ,and could you help me
Thank you --------------------------- Tian Guanhua ----------table schema ----------- Name 'sms' byte[] family1 = Bytes.toBytes("smsdata"); byte[] family2 = Bytes.toBytes("smsdata"); byte[] col1 = Bytes.toBytes("smsdata"); byte[] col2 = Bytes.toBytes("dndata"); --------- code -------------- package com.pattek.sms.hb;
import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.hbase.HBaseConfiguration; import org.apache.hadoop.hbase.KeyValue; import org.apache.hadoop.hbase.client.Get; import org.apache.hadoop.hbase.client.Scan; import org.apache.hadoop.hbase.client.HTable; import org.apache.hadoop.hbase.client.Put; import org.apache.hadoop.hbase.client.Result; import org.apache.hadoop.hbase.client.ResultScanner; import org.apache.hadoop.hbase.util.Bytes; import org.apache.hadoop.hbase.client.coprocessor.AggregationClient;
import org.apache.log4j.Logger; import org.apache.log4j.PropertyConfigurator;
import com.pattek.sms.SmsBase;
public class HbOpr1 extends SmsBase { public static void main(String[] args){ // TODO Auto-generated method stub
Configuration hbase_config = new Configuration(); hbase_config.set("hbase.zookeeper.quorum", "192.168.76.233"); HTable table = null; //System.out.println("hello"); try { // log.info(String.format("Connect To HBase %s", args)); table = new HTable(HBaseConfiguration.create(hbase_config), "sms"); System.out.println("Connect To HBase Completely"); } catch (Exception e1) { e1.printStackTrace(); System.out.println("wrong1"); //SetError(); }
byte[] TABLE_NAME = Bytes.toBytes("sms"); byte[] CF = Bytes.toBytes("smsdata"); //byte[] CF1 = Bytes.toBytes("smsdata"); Configuration configuration HBaseConfiguration.create();
configuration.setLong("hbase.client.scanner.caching", 1000); AggregationClient aggregationClient = new AggregationClient( configuration); Scan scan = new Scan(); byte[] startrow Bytes.toBytes("20121221040000000000009651550307"); byte[] endrow Bytes.toBytes("20121221040000000000009651550308"); scan.setStartRow(startrow); scan.setStopRow(endrow); //scan.addColumn(CF, Bytes.toBytes("smsdata")); scan.addFamily(CF); scan.addColumn(CF, Bytes.toBytes("smsdata")); //System.out.println("**************"); long startime=System.currentTimeMillis(); try{
System.out.println("row count is " + aggregationClient.rowCount(TABLE_NAME, null, scan));
} catch (Throwable e3) { e3.printStackTrace(); System.out.println("wrong3"); } long endtime=System.currentTimeMillis(); System.out.println("\rElapsed time:"+(endtime-startime)+" mini seconds ");
} } -------- code --------------
------------ERROR ---------------- Jul 11, 2012 9:49:32 AM org.apache.zookeeper.Environment logEnv INFO: Client environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT Jul 11, 2012 9:49:32 AM org.apache.zookeeper.Environment logEnv INFO: Client environment:host.name=fnode2 Jul 11, 2012 9:49:32 AM org.apache.zookeeper.Environment logEnv INFO: Client environment:java.version=1.7.0_09 Jul 11, 2012 9:49:32 AM org.apache.zookeeper.Environment logEnv INFO: Client environment:java.vendor=Oracle Corporation Jul 11, 2012 9:49:32 AM org.apache.zookeeper.Environment logEnv INFO: Client environment:java.home=/home/jdk1.7.0_09/jre Jul 11, 2012 9:49:32 AM org.apache.zookeeper.Environment logEnv INFO: Client environment:java.class.path=/home/tianguanhua1/smslx/bin:/home/tianguanhua1/ smslx/lib/apache-solr-core-4.0.0.jar:/home/tianguanhua1/smslx/lib/apache-sol r-solrj-4.0.0.jar:/home/tianguanhua1/smslx/lib/mchange-commons-java-0.2.3.ja r:/home/tianguanhua1/smslx/lib/c3p0-0.9.2-pre5.jar:/home/tianguanhua1/smslx/ lib/c3p0-oracle-thin-extras-0.9.2-pre5.jar:/home/tianguanhua1/smslx/lib/comm ons-beanutils-1.7.0.jar:/home/tianguanhua1/smslx/lib/commons-beanutils-core- 1.8.0.jar:/home/tianguanhua1/smslx/lib/commons-cli-1.2.jar:/home/tianguanhua 1/smslx/lib/commons-codec-1.4.jar:/home/tianguanhua1/smslx/lib/commons-colle ctions-3.2.1.jar:/home/tianguanhua1/smslx/lib/commons-configuration-1.6.jar: /home/tianguanhua1/smslx/lib/commons-daemon-1.0.1.jar:/home/tianguanhua1/sms lx/lib/commons-dbutils-1.5.jar:/home/tianguanhua1/smslx/lib/commons-digester -1.8.jar:/home/tianguanhua1/smslx/lib/commons-el-1.0.jar:/home/tianguanhua1/ smslx/lib/commons-httpclient-3.0.1.jar:/home/tianguanhua1/smslx/lib/commons- io-2.1.jar:/home/tianguanhua1/smslx/lib/commons-lang-2.4.jar:/home/tianguanh ua1/smslx/lib/commons-logging-1.1.1.jar:/home/tianguanhua1/smslx/lib/commons -logging-api-1.0.4.jar:/home/tianguanhua1/smslx/lib/commons-math-2.1.jar:/ho me/tianguanhua1/smslx/lib/commons-net-3.1.jar:/home/tianguanhua1/smslx/lib/c ommons-pool-1.6.jar:/home/tianguanhua1/smslx/lib/guava-r09-jarjar.jar:/home/ tianguanhua1/smslx/lib/hadoop-core-1.1.1.jar:/home/tianguanhua1/smslx/lib/hb ase-0.94.3.jar:/home/tianguanhua1/smslx/lib/httpclient-4.1.3.jar:/home/tiang uanhua1/smslx/lib/httpcore-4.1.4.jar:/home/tianguanhua1/smslx/lib/httpmime-4 .1.3.jar:/home/tianguanhua1/smslx/lib/jcl-over-slf4j-1.6.4.jar:/home/tiangua nhua1/smslx/lib/jna.jar:/home/tianguanhua1/smslx/lib/junit-4.11-beta-1.jar:/ home/tianguanhua1/smslx/lib/log4j-1.2.14.jar:/home/tianguanhua1/smslx/lib/lo g4j-over-slf4j-1.6.4.jar:/home/tianguanhua1/smslx/lib/lucene-analyzers-commo n-4.0.0.jar:/home/tianguanhua1/smslx/lib/lucene-core-4.0.0.jar:/home/tiangua nhua1/smslx/lib/lucene-queryparser-4.0.0.jar:/home/tianguanhua1/smslx/lib/my sql-connector-java-5.1.12-bin.jar:/home/tianguanhua1/smslx/lib/mysql-connect or-java-5.1.22-bin.jar:/home/tianguanhua1/smslx/lib/ojdbc14.jar:/home/tiangu anhua1/smslx/lib/protobuf-java-2.4.1.jar:/home/tianguanhua1/smslx/lib/slf4j- api-1.6.4.jar:/home/tianguanhua1/smslx/lib/slf4j-jdk14-1.6.4.jar:/home/tiang uanhua1/smslx/lib/zookeeper-3.4.5.
-
答复: how to use API to statistic how many message has been store in the table in hbase
tgh 2012-12-27, 07:21
Meanwhile , I see , the AggregationClient#rowCount() is to statistic the Number of given Family and given col in a table, and it scan the related region to satistic the Number, that is, some colume can be null, is it,
But If I want to know the row number of table, no matter have data there, I just want to know the row number of the whole table, and it is a large table, And which API should I use, and how to use it,
Could you help me
Thank you --------------------- Tian Guanhua
-----邮件原件----- 发件人: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] 代表 Anoop Sam John 发送时间: 2012年12月27日 13:31 收件人: [EMAIL PROTECTED] 主题: RE: how to use API to statistic how many message has been store in the table in hbase
So you want to know the no# of rows in a table? Have a look at AggregationClient#rowCount()
-Anoop- ________________________________________ From: tgh [[EMAIL PROTECTED]] Sent: Thursday, December 27, 2012 7:51 AM To: [EMAIL PROTECTED] Subject: how to use API to statistic how many message has been store in the table in hbase
Hi I try to use hbaseAPI to store data, and I want to get the number of the message stored in the table in HBase, how to use API to statistic how many message has been store in the table in hbase What API can do this, Could you help me Thank you --------------------- Tian Guanhua
|
|