Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Problem with Hive/HBase integration


Copy link to this message
-
Re: Problem with Hive/HBase integration
http://dl.dropbox.com/u/19454506/HadoopHIveHbaseReady.tar.gz
Download this its pre connfigured hive and hbase. you need to change some
settings accordingly specific to your linux settings...

On Fri, Jan 27, 2012 at 11:07 AM, Madhusudhana Rao Podila <
[EMAIL PROTECTED]> wrote:

> Hi****
>
> ** **
>
> I have a problem in create a Hive table using existing HBase table (using
> External Table concept) with multiple columns from column family (not using
> as Map)****
>
> ** **
>
> *Case-1 :*
>
> I have created a table in HBase and able to map to Hive as an external
> table just using only one column from the column family ****
>
> ** **
>
> HBase ****
>
> Created the table in HBase using the following command****
>
>                 ****
>
> hbase(main):001:0> create 'hbasetohive', 'colfamily'****
>
> 0 row(s) in 1.9700 seconds****
>
> ** **
>
> hbase(main):002:0> put 'hbasetohive', '1s', 'colfamily:val','1strowval'***
> *
>
> 0 row(s) in 0.2240 seconds****
>
> ** **
>
> hbase(main):003:0> scan 'hbasetohive'****
>
> ROW
> COLUMN+CELL                                                  ****
>
>  1s                    column=colfamily:val, timestamp=1327676987075,
> value=1strowva****
>
>
> l                                                            ****
>
> 1 row(s) in 0.0840 seconds****
>
> ** **
>
> *Hive*
>
> ** **
>
> hive> CREATE EXTERNAL TABLE hbase_hivetable_k(key string, value string)***
> *
>
>     > STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'****
>
>     > WITH SERDEPROPERTIES("hbase.columns.mapping" = "colfamily:val")****
>
>     > TBLPROPERTIES("hbase.table.name" = "hbasetohive");****
>
> OK****
>
> Time taken: 10.808 seconds****
>
> hive> select * from hbase_hivetable_k;****
>
> OK****
>
> 1s      1strowval****
>
> Time taken: 1.314 seconds****
>
> ** **
>
> *Case 2*
>
> ** **
>
> I have created a table in HBase with column family as cf_cdr with two
> columns caller_name, caller_number; Then I tried creating the Hive table
> using the HBase table that got created by specifying both columns from the
> column family,  It is throwing Metaexteception: If I restrict to only one
> column am able to create the table in Hive properly****
>
> ** **
>
> *HBase*
>
> hbase(main):004:0> create 'hb_cdr', 'cf_cdr'****
>
> 0 row(s) in 1.4870 seconds****
>
> ** **
>
> hbase(main):005:0> put 'hb_cdr', 'cdr_r1', 'cf_cdr:caller_name', 'madhu'**
> **
>
> 0 row(s) in 0.0490 seconds****
>
> ** **
>
> hbase(main):006:0> put 'hb_cdr', 'cdr_r1', 'cf_cdr:caller_number',
> '08877232010'****
>
> 0 row(s) in 0.0300 seconds****
>
> ** **
>
> hbase(main):007:0> put 'hb_cdr', 'cdr_r2', 'cf_cdr:caller_name', 'bharat'*
> ***
>
> 0 row(s) in 0.0170 seconds****
>
> ** **
>
> hbase(main):008:0> scan 'hb_cdr'****
>
> ROW
> COLUMN+CELL                                                  ****
>
>  cdr_r1                column=cf_cdr:caller_name, timestamp=1327677898993,
> value=mad****
>
>
> hu                                                           ****
>
>  cdr_r1                column=cf_cdr:caller_number,
> timestamp=1327677912648, value=0****
>
>
>               8877232010
> ****
>
>  cdr_r2                column=cf_cdr:caller_name, timestamp=1327677919720,
> value=bha****
>
>
> rat                                                          ****
>
> 2 row(s) in 0.1020 seconds****
>
> ** **
>
> *Hive*
>
> * *
>
> hive> CREATE EXTERNAL TABLE hv_hb_cdr(key string, c_name string, c_number
> string)****
>
>     > STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'****
>
>     > WITH SERDEPROPERTIES("hbase.columns.mapping" = "cf_cdr:caller_name,
> cf_cdr:caller_number")****
>
>     > TBLPROPERTIES("hbase.table.name" = "hb_cdr");****
>
> FAILED: Error in metadata: MetaException(message:Column Family  cf_cdr is
> not defined in hbase table hb_cdr)****
>
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask****
>
> ** **
>
> Is there anything issue in the above script?****
>
> ** **
>
> Please suggest****
Shashwat Shriparv
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB