Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # user - Problem with Hive/HBase integration


Copy link to this message
-
Re: Problem with Hive/HBase integration
shashwat shriparv 2012-01-27, 05:47
http://dl.dropbox.com/u/19454506/HadoopHIveHbaseReady.tar.gz
Download this its pre connfigured hive and hbase. you need to change some
settings accordingly specific to your linux settings...

On Fri, Jan 27, 2012 at 11:07 AM, Madhusudhana Rao Podila <
[EMAIL PROTECTED]> wrote:

> Hi****
>
> ** **
>
> I have a problem in create a Hive table using existing HBase table (using
> External Table concept) with multiple columns from column family (not using
> as Map)****
>
> ** **
>
> *Case-1 :*
>
> I have created a table in HBase and able to map to Hive as an external
> table just using only one column from the column family ****
>
> ** **
>
> HBase ****
>
> Created the table in HBase using the following command****
>
>                 ****
>
> hbase(main):001:0> create 'hbasetohive', 'colfamily'****
>
> 0 row(s) in 1.9700 seconds****
>
> ** **
>
> hbase(main):002:0> put 'hbasetohive', '1s', 'colfamily:val','1strowval'***
> *
>
> 0 row(s) in 0.2240 seconds****
>
> ** **
>
> hbase(main):003:0> scan 'hbasetohive'****
>
> ROW
> COLUMN+CELL                                                  ****
>
>  1s                    column=colfamily:val, timestamp=1327676987075,
> value=1strowva****
>
>
> l                                                            ****
>
> 1 row(s) in 0.0840 seconds****
>
> ** **
>
> *Hive*
>
> ** **
>
> hive> CREATE EXTERNAL TABLE hbase_hivetable_k(key string, value string)***
> *
>
>     > STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'****
>
>     > WITH SERDEPROPERTIES("hbase.columns.mapping" = "colfamily:val")****
>
>     > TBLPROPERTIES("hbase.table.name" = "hbasetohive");****
>
> OK****
>
> Time taken: 10.808 seconds****
>
> hive> select * from hbase_hivetable_k;****
>
> OK****
>
> 1s      1strowval****
>
> Time taken: 1.314 seconds****
>
> ** **
>
> *Case 2*
>
> ** **
>
> I have created a table in HBase with column family as cf_cdr with two
> columns caller_name, caller_number; Then I tried creating the Hive table
> using the HBase table that got created by specifying both columns from the
> column family,  It is throwing Metaexteception: If I restrict to only one
> column am able to create the table in Hive properly****
>
> ** **
>
> *HBase*
>
> hbase(main):004:0> create 'hb_cdr', 'cf_cdr'****
>
> 0 row(s) in 1.4870 seconds****
>
> ** **
>
> hbase(main):005:0> put 'hb_cdr', 'cdr_r1', 'cf_cdr:caller_name', 'madhu'**
> **
>
> 0 row(s) in 0.0490 seconds****
>
> ** **
>
> hbase(main):006:0> put 'hb_cdr', 'cdr_r1', 'cf_cdr:caller_number',
> '08877232010'****
>
> 0 row(s) in 0.0300 seconds****
>
> ** **
>
> hbase(main):007:0> put 'hb_cdr', 'cdr_r2', 'cf_cdr:caller_name', 'bharat'*
> ***
>
> 0 row(s) in 0.0170 seconds****
>
> ** **
>
> hbase(main):008:0> scan 'hb_cdr'****
>
> ROW
> COLUMN+CELL                                                  ****
>
>  cdr_r1                column=cf_cdr:caller_name, timestamp=1327677898993,
> value=mad****
>
>
> hu                                                           ****
>
>  cdr_r1                column=cf_cdr:caller_number,
> timestamp=1327677912648, value=0****
>
>
>               8877232010
> ****
>
>  cdr_r2                column=cf_cdr:caller_name, timestamp=1327677919720,
> value=bha****
>
>
> rat                                                          ****
>
> 2 row(s) in 0.1020 seconds****
>
> ** **
>
> *Hive*
>
> * *
>
> hive> CREATE EXTERNAL TABLE hv_hb_cdr(key string, c_name string, c_number
> string)****
>
>     > STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'****
>
>     > WITH SERDEPROPERTIES("hbase.columns.mapping" = "cf_cdr:caller_name,
> cf_cdr:caller_number")****
>
>     > TBLPROPERTIES("hbase.table.name" = "hb_cdr");****
>
> FAILED: Error in metadata: MetaException(message:Column Family  cf_cdr is
> not defined in hbase table hb_cdr)****
>
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask****
>
> ** **
>
> Is there anything issue in the above script?****
>
> ** **
>
> Please suggest****
Shashwat Shriparv