Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Re: create external table in CLI


Copy link to this message
-
Re: create external table in CLI
Adding hive-user.

On Thu, Mar 25, 2010 at 9:38 PM, Ted Yu <[EMAIL PROTECTED]> wrote:

> If I use the following commandline, I would see RetriesExhaustedException
> as I reported earlier this afternoon:
> [root@tyu-linux dist]# bin/hive -hiveconf hbase.master=snv-it-lin-006
> -hiveconf
> hbase.zookeeper.quorum=snv-it-lin-010,snv-it-lin-011,snv-it-lin-012
> -hiveconf hbase.zookeeper.property.clientPort=2181
>
> org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to contact
> region server 10.10.31.136:60020 for region .META.,,1, row '', but failed
> after 10 attempts.
>
> On Thu, Mar 25, 2010 at 9:18 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
>> I searched source code tree under trunk and didn't see how hbase.master
>> parameter is passed to HBaseStorageHandler.java
>>
>> Also it looks like the unit test only specifies hostname for hbase.master:
>> "Z:\kindsight\hive\trunk\hbase-handler\src\test\org\apache\hadoop\hive\hbase\HBaseQTestUtil.java"(55,17):
>> conf.set("hbase.master", "local");
>>
>> BTW I saw a lot of following exceptions:
>> 2010-03-25 13:24:27,056 WARN  zookeeper.ClientCnxn
>> (ClientCnxn.java:cleanup(1001)) - Ignoring exception during shutdown input
>> java.nio.channels.ClosedChannelException
>>         at
>> sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:638)
>>         at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:360)
>>         at
>> org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:999)
>>         at
>> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:970)
>>
>> Thanks
>>
>>
>> On Thu, Mar 25, 2010 at 7:12 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>>
>>> We use distributed mode because the cluster would be deployed to
>>> production.
>>>
>>> I didn't find zookeeper commandline parameter on
>>> http://wiki.apache.org/hadoop/Hive/HBaseIntegration#Usage
>>> Do I need to use an entry in hbase-site.xml ?
>>>
>>> Our hbase cluster is based on 0.20.1
>>> But I used hbase client 0.20.3 against it before and didn't see
>>> RetriesExhaustedException.
>>>
>>> Thanks
>>>
>>>
>>> On Thu, Mar 25, 2010 at 6:07 PM, John Sichi <[EMAIL PROTECTED]> wrote:
>>>
>>>> Hmmm, what kind of HBase cluster are you running (standalone,
>>>> pseudo-distributed, or distributed)?  If distributed, you may need to pass
>>>> zookeeper connection info instead of hbase.master; I haven't tested out
>>>> distributed yet but will be soon.
>>>>
>>>> JVS
>>>>
>>>> On Mar 25, 2010, at 4:43 PM, Ted Yu wrote:
>>>>
>>>> I removed hbase-site.xml
>>>>
>>>> [root@tyu-linux dist]# bin/hive -hiveconf
>>>> hbase.master=snv-it-lin-006:60000
>>>> Hive history
>>>> file=/tmp/root/hive_job_log_root_201003251614_1493474415.txt
>>>> hive> CREATE EXTERNAL TABLE ruletable(key string, exactmatch_cat string,
>>>> lpm_cat int)
>>>>     >     STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
>>>>     >     WITH SERDEPROPERTIES (
>>>>     >     "hbase.key.type" = "string",
>>>>     >     "hbase.columns.mapping" >>>> "exactmatch_1.0:category,lpm_1.0:category",
>>>>     >     "hbase.table.name" = "ruletable"
>>>>     >     );
>>>> FAILED: Error in metadata:
>>>> MetaException(message:org.apache.hadoop.hbase.MasterNotRunningException
>>>>         at
>>>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getMaster(HConnectionManager.java:374)
>>>>         at
>>>> org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:72)
>>>>         at
>>>> org.apache.hadoop.hive.hbase.HBaseStorageHandler.getHBaseAdmin(HBaseStorageHandler.java:63)
>>>>         at
>>>> org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:149)
>>>>         at
>>>> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:280)
>>>>         at
>>>> org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:320)
>>>>         at
>>>> org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:1445)
>>>>         at
>>>> org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:124)
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB