Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive >> mail # user >> Custom hive-site.xml is ignored, how to find out why


+
Stephen Boesch 2012-11-24, 22:19
+
Stephen Boesch 2012-11-25, 04:41
+
Shreepadma Venugopalan 2012-11-26, 19:47
Copy link to this message
-
Re: Custom hive-site.xml is ignored, how to find out why
Hi,
  The problem may not have to do with hive-site.xml.  When I run hive
client by itself it connects successfully to mysql and creates / reads
metadataa.

The problem comes in when i run the metastore/thrift servers via:   hive
--service metastore   hive --service hiveserver.   As soon as i do that,
and try to run hive client (hive cli) then it goes back to trying to use
derby.

There must be additional settings required for the hive metastore service
beyond those in my hive-site.xml.

Btw from OP i aleady have the settings in hive-site.xml for connecting via
jdo to mysql.    I don't have the data nucleus ones. Are those required?
The docs i saw did not mention them.
2012/11/26 Shreepadma Venugopalan <[EMAIL PROTECTED]>

> Hi Stephen,
>
> If you wish to setup a mysql metastore, you need to have the following in
> your hive-site.xml,
>
> <property>
>    <name>javax.jdo.option.ConnectionURL</name>
>    <value>jdbc:mysql://MYHOST/metastore</value>
>  </property>
>
> <property>
>    <name>javax.jdo.option.ConnectionDriverName</name>
>    <value>com.mysql.jdbc.Driver</value>
>  </property>
>
> <property>
>    <name>javax.jdo.option.ConnectionUserName</name>
>    <value>hiveuser</value>
>  </property>
>
> <property>
>    <name>javax.jdo.option.ConnectionPassword</name>
>    <value>password</value>
>  </property>
>
> <property>
>    <name>datanucleus.autoCreateSchema</name>
>    <value>false</value>
>  </property>
>
> <property>
>    <name>datanucleus.fixedDatastore</name>
>    <value>true</value>
>  </property>
>
>
> Thanks.
> Shreepadma
>
>
> On Sat, Nov 24, 2012 at 8:41 PM, Stephen Boesch <[EMAIL PROTECTED]> wrote:
>
>> It appears that I were missing the *hive.metastore.uris* parameter.
>>  That one was not mentioned in the (several) blogs / tutorials that I had
>> seen.
>>
>>
>> 2012/11/24 Stephen Boesch <[EMAIL PROTECTED]>
>>
>>>
>>> It seems the customized hive-site.xml is not being read. It lives under
>>> $HIVE_HOME/conf  ( which happens to be /shared/hive/conf).  I have tried
>>> everything there is to try:  set HIVE_CONF_DIR=/shared/hive/conf , added
>>> --config /shared/hive/conf  and added debugging to the hive shell script
>>> (bash -x) to ensure that the correct directory is actually getting used
>>>
>>> The properties inside hive-site.xml are getting ignored: originally set
>>> to using mysql , but instead the default derby is getting used. Then tried
>>> changing  hive.metastore.local between true and false: no difference in
>>> behavior - just going to /tmp/$USER for creating the derby no matter what.
>>>
>>> I wondered whether hive-site.xml maybe has a syntax error and were
>>> getting ignored: so removed everything except
>>> <configuration></configuration>:  still no
>>>
>>> This is the totally simplified hive-site.xml: it just has enough to try
>>> to see if it is actually being read/applied (and it is not..):
>>>
>>>
>>> steve@mithril:/shared/hive/conf$ cat hive-site.xml
>>> <?xml version="1.0"?>
>>> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
>>>
>>> <configuration>
>>>
>>> <!-- Hive Execution Parameters -->
>>> <property>
>>>   <name>hive.metastore.local</name>
>>>   <value>false</value>
>>> </property>
>>>
>>> <property>
>>> <name>hive.exec.scratchdir</name>
>>> <value>/tmp/hive/hive-${user.name}</value>
>>> <description>Scratch space for Hive jobs</description>
>>> </property>
>>>
>>> <property>
>>>   <name>hive.hwi.war.file</name>
>>>   <value>/shared/hive/lib/hive-hwi-0.9.0.war</value>
>>>   <description>This is the WAR file with the jsp content for Hive Web
>>> Interface</description>
>>> </property>
>>>
>>> </configuration>
>>>
>>>
>>> After running some DDL in hive, for example, no files are created
>>> underneath /tmp/hive  (instead they are going to /tmp/$USER which is
>>> default - as if the custom hive-site.xml never existed.
>>>
>>>
>>>
>>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB