Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Hive-Hbase integration

Copy link to this message
Re: Hive-Hbase integration
Hi Bejoy,

   Thank you so much for your help again..Your way of explaining
things is really great..And the query provided by you is working
absolutely fine.

    Mohammad Tariq

On Sun, Dec 18, 2011 at 12:35 AM, Bejoy Ks <[EMAIL PROTECTED]> wrote:
> Hi Tariq
>     From the stack trace, I believe the issue could be due to the fact that
> you are just providing Column Families but no Qualifiers in the
> hbase.columns.mapping. If you don't specify the qualifier for a column
> family then the hive column would be mapped to all the Qualifiers
> corresponding to that hbase Column Family. So here what happens is that ,all
> the qualifiers for each column family is made to map and this map is
> supposed to be stored in hive tables, but in your query you are mapping
> these maps to primitives and it results in the exception. In hive wiki such
> an operation is mentioned illegal, please refer
> https://cwiki.apache.org/Hive/hbaseintegration.html#HBaseIntegration-ColumnMapping
> https://cwiki.apache.org/Hive/hbaseintegration.html#HBaseIntegration-Illegal%253AHivePrimitivetoHBaseColumnFamily
> You can get your query working by just changing the data type of Hbase
> columns also better to add key in your mapping,
> CREATE EXTERNAL TABLE employee(key string,no map<string,string>,name
> map<string,string>,address map<string,string>)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,no:,name:,address:")
> TBLPROPERTIES("hbase.table.name"= "employee");
> For your second question, In Hbase every row is uniquely identified by the
> ROW_KEY, here with :key in our mapping we are mapping this row key to one of
> our hive table column. From your output the two values of row key in hbase
> employee table are emp1 and emp2. I believe your confusion is from the hbase
> CLI output. In RDBMS/hive query we see a record in a line on querying, but
> in hbase shell one line represents a column family not an entire record
> unlike hive. If there are 10 column families in your hbase table, then on
> scan command you get 10 lines for one record (by record in Hbase i refer to
> all the attributes corresponding to a row key). Here since you have 3 Column
> Families, you see 3 lines represent a record(attributes of emp*) .
> Hope it helps!...
> Regards
> Bejoy.K.S
> ________________________________
> From: Mohammad Tariq <[EMAIL PROTECTED]>
> To: user <[EMAIL PROTECTED]>
> Sent: Saturday, December 17, 2011 11:32 PM
> Subject: Hive-Hbase integration
> Hello list,
>     I have a small demo table in Hbase and I want to operate it
> through Hive.Here is my table in Hbase -
> hbase(main):021:0> scan 'employee'
> ROW                                    COLUMN+CELL
> emp1                                  column=address:,
> timestamp=1324119715536, value=#12-bangalore
> emp1                                  column=name:,
> timestamp=1324119698581, value=tariq
> emp1                                  column=no:,
> timestamp=1324119688511, value=001
> emp2                                  column=address:,
> timestamp=1324120893996, value=#13-bangalore
> emp2                                  column=name:,
> timestamp=1324120883612, value=vishal
> emp2                                  column=no:,
> timestamp=1324120866981, value=002
> 2 row(s) in 0.0260 seconds
> I have 2 rows in the employee table, each corresponding to a
> particular user. And I have 3 column families (each having only 1
> column) - no, name and address.
> For this table I have created an external table in Hive using the
> following command -
> hive> CREATE EXTERNAL TABLE employee(key string,no string,name
> string,address string)    > STORED BY
> 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
>   > WITH SERDEPROPERTIES ("hbase.columns.mapping" > "no:,name:,address:")              > TBLPROPERTIES("hbase.table.name"
> = "employee");
> But i am getting the following error -
> FAILED: Error in metadata: java.lang.RuntimeException: