Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # user - Re: create external table in CLI


Copy link to this message
-
Re: create external table in CLI
Ted Yu 2010-03-26, 04:39
Adding hive-user.

On Thu, Mar 25, 2010 at 9:38 PM, Ted Yu <[EMAIL PROTECTED]> wrote:

> If I use the following commandline, I would see RetriesExhaustedException
> as I reported earlier this afternoon:
> [root@tyu-linux dist]# bin/hive -hiveconf hbase.master=snv-it-lin-006
> -hiveconf
> hbase.zookeeper.quorum=snv-it-lin-010,snv-it-lin-011,snv-it-lin-012
> -hiveconf hbase.zookeeper.property.clientPort=2181
>
> org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to contact
> region server 10.10.31.136:60020 for region .META.,,1, row '', but failed
> after 10 attempts.
>
> On Thu, Mar 25, 2010 at 9:18 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
>> I searched source code tree under trunk and didn't see how hbase.master
>> parameter is passed to HBaseStorageHandler.java
>>
>> Also it looks like the unit test only specifies hostname for hbase.master:
>> "Z:\kindsight\hive\trunk\hbase-handler\src\test\org\apache\hadoop\hive\hbase\HBaseQTestUtil.java"(55,17):
>> conf.set("hbase.master", "local");
>>
>> BTW I saw a lot of following exceptions:
>> 2010-03-25 13:24:27,056 WARN  zookeeper.ClientCnxn
>> (ClientCnxn.java:cleanup(1001)) - Ignoring exception during shutdown input
>> java.nio.channels.ClosedChannelException
>>         at
>> sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:638)
>>         at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:360)
>>         at
>> org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:999)
>>         at
>> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:970)
>>
>> Thanks
>>
>>
>> On Thu, Mar 25, 2010 at 7:12 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>>
>>> We use distributed mode because the cluster would be deployed to
>>> production.
>>>
>>> I didn't find zookeeper commandline parameter on
>>> http://wiki.apache.org/hadoop/Hive/HBaseIntegration#Usage
>>> Do I need to use an entry in hbase-site.xml ?
>>>
>>> Our hbase cluster is based on 0.20.1
>>> But I used hbase client 0.20.3 against it before and didn't see
>>> RetriesExhaustedException.
>>>
>>> Thanks
>>>
>>>
>>> On Thu, Mar 25, 2010 at 6:07 PM, John Sichi <[EMAIL PROTECTED]> wrote:
>>>
>>>> Hmmm, what kind of HBase cluster are you running (standalone,
>>>> pseudo-distributed, or distributed)?  If distributed, you may need to pass
>>>> zookeeper connection info instead of hbase.master; I haven't tested out
>>>> distributed yet but will be soon.
>>>>
>>>> JVS
>>>>
>>>> On Mar 25, 2010, at 4:43 PM, Ted Yu wrote:
>>>>
>>>> I removed hbase-site.xml
>>>>
>>>> [root@tyu-linux dist]# bin/hive -hiveconf
>>>> hbase.master=snv-it-lin-006:60000
>>>> Hive history
>>>> file=/tmp/root/hive_job_log_root_201003251614_1493474415.txt
>>>> hive> CREATE EXTERNAL TABLE ruletable(key string, exactmatch_cat string,
>>>> lpm_cat int)
>>>>     >     STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
>>>>     >     WITH SERDEPROPERTIES (
>>>>     >     "hbase.key.type" = "string",
>>>>     >     "hbase.columns.mapping" >>>> "exactmatch_1.0:category,lpm_1.0:category",
>>>>     >     "hbase.table.name" = "ruletable"
>>>>     >     );
>>>> FAILED: Error in metadata:
>>>> MetaException(message:org.apache.hadoop.hbase.MasterNotRunningException
>>>>         at
>>>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getMaster(HConnectionManager.java:374)
>>>>         at
>>>> org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:72)
>>>>         at
>>>> org.apache.hadoop.hive.hbase.HBaseStorageHandler.getHBaseAdmin(HBaseStorageHandler.java:63)
>>>>         at
>>>> org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:149)
>>>>         at
>>>> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:280)
>>>>         at
>>>> org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:320)
>>>>         at
>>>> org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:1445)
>>>>         at
>>>> org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:124)