Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # user - Problem when trying to connect to hive server using jdbc


Copy link to this message
-
Re: Problem when trying to connect to hive server using jdbc
Sanjay Subramanian 2013-04-03, 01:55
Try the following…I am hoping you have run the hive MYSQL SOURCE script to create the metastore tables and created hive_user  with appropriate permissions

Change localhost to FQDN
<property>
  <name>javax.jdo.option.ConnectionURL</name>
  <value>jdbc:mysql://FQDN/metastore?createDatabaseIfNotExist=false</value>
  <description>JDBC connect string for a JDBC metastore</description>
</property>

Change port from 3306 to 9083
<property>
  <name>hive.metastore.uris</name>
  <value>thrift://my_ipaddress:3306</value>
  <description>IP address (or fully-qualified domain name) and port of the metastore host</description>
</property>
From: demian rosas <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Reply-To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Date: Tuesday, April 2, 2013 6:48 PM
To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Subject: Re: Problem when trying to connect to hive server using jdbc

My linux version is

Linux myhost 2.6.18-308.4.1.0.1.el5xen #1 SMP Tue Apr 17 16:41:30 EDT 2012 x86_64 x86_64 x86_64 GNU/Linux

which is some version of Oracle linux which is supposed to be Red Hat compatible

> echo $HADOOP_MAPRED_HOME
/usr/lib/hadoop-0.20-mapreduce

> echo $HIVE_HOME
/usr/lib/hive/

These variables see to be fine.

Just a little bit more of context:

I am trying to set a hive installation using a remote mysql metastore. I am using CDH4.2 on a fresh installation. All this in a single machine, so I am using hadoop pseudo distributed mode.

So far I have Hadoop working fine. MySql is working and I can connect to it using jdbc from a java application. When I installed hive for the first time, it was using the embedded mode and I was able to define an external table pointing to an hdfs location and query the data with a hive query. Until here everything was fine.

The problems started when I tried to set the remote mysql metastore. I have followed the instructions provided here:  https://ccp.cloudera.com/display/CDH4DOC/Hive+Installation#HiveInstallation-ConfiguringtheHiveMetastore

Right now I am using hive-server 1 configuration. This are the properties in my hive-site.xml file:

<property>
  <name>javax.jdo.option.ConnectionURL</name>
  <value>jdbc:mysql://localhost/metastore?createDatabaseIfNotExist=true</value>
  <description>JDBC connect string for a JDBC metastore</description>
</property>

<property>
  <name>javax.jdo.option.ConnectionDriverName</name>
  <value>com.mysql.jdbc.Driver</value>
  <description>Driver class name for a JDBC metastore</description>
</property>

<property>
  <name>javax.jdo.option.ConnectionUserName</name>
  <value>myuser</value>
</property>

<property>
  <name>javax.jdo.option.ConnectionPassword</name>
  <value>mypassword</value>
</property>

<property>
  <name>datanucleus.autoCreateSchema</name>
  <value>false</value>
</property>

<property>
  <name>datanucleus.fixedDatastore</name>
  <value>true</value>
</property>

<property>
  <name>hive.metastore.uris</name>
  <value>thrift://my_ipaddress:3306</value>
  <description>IP address (or fully-qualified domain name) and port of the metastore host</description>
</property>

<property>
  <name>hive.metastore.warehouse.dir</name>
  <value>/user/hive/warehouse</value>
</property>

<property>
  <name>hive.server.thrift.port</name>
   <value>10000</value>
</property>
On 2 April 2013 18:36, Sanjay Subramanian <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
I just modified the hive batch file to echo some variables. These are what the variables are set to when I execute /usr/lib/hive/bin/hive

HADOOP=/usr/bin/../bin/hadoop
HADOOP_HOME=/usr/bin/..
HADOOP_VERSION=2.0.0-cdh4.1.2

R these variables set correctly for u  ?
HADOOP_MAPRED_HOME=/usr/lib/hadoop-mapreduce
HIVE_HOME=/usr/lib/hive

Which linux  are u running ?

From: demian rosas <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>Reply-To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Date: Tuesday, April 2, 2013 6:18 PM

To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Subject: Re: Problem when trying to connect to hive server using jdbc

The content of the indicated file on line 179 "area" is

# Save the regex to a var to workaround quoting incompatabilities
# between Bash 3.1 and 3.2
hadoop_version_re="^([[:digit:]]+)\.([[:digit:]]+)(\.([[:digit:]]+))?.*$"

if [[ "$HADOOP_VERSION" =~ $hadoop_version_re ]]; then #--------------------------------------------------------------------------------------> This is line 179
    hadoop_major_ver=${BASH_REMATCH[1]}
    hadoop_minor_ver=${BASH_REMATCH[2]}
    hadoop_patch_ver=${BASH_REMATCH[4]}
else
    echo "Unable to determine Hadoop version information."
    echo "'hadoop version' returned:"
    echo `$HADOOP version`
    exit 5
fi

I do not see anything particularly wrong there.

What do you think?

On 2 April 2013 18:09, Sanjay Subramanian <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
In the hive batch file
There seem to be some errors on 179…can u verify
sanjay

From: demian rosas <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Reply-To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Date: Tuesday, April 2, 2013 5:56 PM

To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Subject: Re: Problem when trying to connect to hive server using jdbc

Yes,

I restart all Hadoop/Hive services every time I change something.

The content of my /var/log/hive/hive-metastore.log file is

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/util/VersionInfo
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.util.VersionInfo
at java.net.URLClassLoader$1