Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - HBase and Hive


Copy link to this message
-
Re: HBase and Hive
Ted Yu 2013-09-17, 17:24
hbase 0.94.6.1 and hbase 0.94.11 are compatible.

Looks like Hive user mailing list would be better place for future
discussion on remaining issues.
On Tue, Sep 17, 2013 at 10:19 AM, Michael Kintzer <[EMAIL PROTECTED]> wrote:

> Many thanks for the assist.  That makes sense.  The map keys represent the
> column-family columns, and the values the column values, right?
>
> I think I've made progress but now am trying to resolve a classpath issue
> with hive that others have run into:
>
> java.io.IOException: Cannot create an instance of InputSplit class > org.apache.hadoop.hive.hbase.HBaseSplit:org.apache.hadoop.hive.hbase.HBaseSplit
>
>
> and
>
> Error: java.lang.ClassNotFoundException:
> org.apache.zookeeper.KeeperException
>
>
> Trying to solve that by setting up $HIVE_HOME/auxlib with the correct jars
> and defining auxlib in hive-site.xml.  I noticed hive 0.11.0 comes with
> hbase-0.94.6.1, but I'm running hbase 0.94.11.  Could that also cause
> incompatibilities?
>
> -Michael
>
>
> ________________________________
>  From: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>; Michael Kintzer <
> [EMAIL PROTECTED]>
> Sent: Monday, September 16, 2013 4:59 PM
> Subject: Re: HBase and Hive
>
>
>
> > "hbase.columns.mapping" = ":key,f:vals"
>
>
> This is where the error is. Instead of "vals", you should have the name of
> the column under that column family. If you want to pull in all the column
> you can simply change it to "f:" and it will pull in all the columns.
> However be sure to change the corresponding hive column to be of a map type
> then.
>
> Hope this helps.
>
>
>
> On Mon, Sep 16, 2013 at 6:33 PM, Michael Kintzer <[EMAIL PROTECTED]>
> wrote:
>
> Hi,
> >
> >Newbie here.
> >
> >hbase 0.94.11
> >hadoop 1.2.1
> >hive 0.11.0
> >
> >I've created an HBase table in hbase shell using command:   create
> 'mytable', 'f'
> >
> >I've loaded data into that table using a thrift Ruby client.   A table
> row has a string key like 'someurl.com:yyyy-mm-dd'.   The column-family
> 'f' has a variable number of columns/cells of data that look like:
> >
> >'f:n'  timestamp=<some timestamp>, value='some JSON',  where n is an
> integer.
> >
> >All this data is viewable in hbase shell and via thrift.
> >
> >I've created a Hive external table pointing to this HBase table with:
> >
> >create external table hbase_mytable (id string, vals ARRAY<string>)
>
> >  STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> >  WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,f:vals")
>
> >  TBLPROPERTIES ("hbase.table.name" = "mytable");
> >
> >
> >This succeeds.  However any Hive QL queries against this table return no
> data.   Even select count(*) from hbase_mytable return 0 records.
> >
> >Anyone have a suggestion for me as to what might be missing from my hive
> installation that prevents it from seeing the data in the HBase table?
> >
> >Much appreciated,
> >
> >Michael
> >
>
>
> --
> Swarnim
>