Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive, mail # user - Mapping existing HBase table with many columns to Hive.


Copy link to this message
-
Mapping existing HBase table with many columns to Hive.
David Koch 2012-12-06, 18:56
Hello,

How can I map an HBase table with the following layout to Hive using the
"CREATE EXTERNAL TABLE" command from shell (or another programmatic way):

The HBase table's layout is as follows:
Rowkey=16 bytes, a UUID that had the "-" removed, and the 32hex chars
converted into two 8byte longs.
Columns (qualifiers): timestamps, i.e the bytes of a long which were
converted using Hadoop's Bytes.toBytes(long). There can be many of those in
a single row.
Values: The bytes of a Java string.

I am unsure of which datatypes to use. I am pretty sure there is no way I
can sensible map the row key to anything other than "binary" but maybe the
columns - which are longs and the values which are strings can be mapped to
their according Hive datatypes.

I include an extract of what a row looks like in HBase shell below:

Thank you,

/David

hbase(main):009:0> scan "hits"
ROW
              COLUMN+CELL

\x00\x00\x06\xB1H\x89N\xC3\xA5\x83\x0F\xDD\x1E\xAE&\xDC
 column=t:\x00\x00\x01;2\xE6Q\x06, timestamp=1267737987733, value=blahaha
\x00\x00\x06\xB1H\x89N\xC3\xA5\x83\x0F\xDD\x1E\xAE&\xDC
 column=t:\x00\x00\x01;2\xE6\xFB@, timestamp=1354012104967, value=testtest
+
kulkarni.swarnim@...) 2012-12-06, 20:10
+
David Koch 2012-12-06, 20:23
+
David Koch 2012-12-10, 00:03
+
kulkarni.swarnim@...) 2012-12-10, 00:52