Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive, mail # user - Re: hive-site.xml not found on classpath


+
Stephen Boesch 2012-11-29, 08:56
+
Shreepadma Venugopalan 2012-11-29, 20:42
+
Stephen Boesch 2012-11-29, 20:46
+
kulkarni.swarnim@...) 2012-11-29, 20:54
+
Stephen Boesch 2012-11-29, 21:02
+
Bing Li 2012-11-30, 07:01
+
Stephen Boesch 2012-11-30, 07:05
+
Bing Li 2012-11-30, 08:01
+
Stephen Boesch 2012-11-30, 08:15
+
Lauren Yang 2012-11-30, 18:46
+
Stephen Boesch 2012-12-09, 08:06
Copy link to this message
-
Re: hive-site.xml not found on classpath
Stephen Boesch 2012-12-09, 20:32
I ended up patching the HiveConf.java  . If  hive-site.xml were not found
on the classpath then:

   - an   o.a.h.fs.Path object is created from
    System.getenv("HIVE_CONF_DIR") + File.seperator + "hive-site.xml"
   - the Path is sent to the base class Configuration.addResource - whiich
   btw accepts either resourceURL's or Path objects

the problem was resolved  with that change to HiveConf.java:  the
hive-site.xml is loaded properly via the Path (instead of URL)  and as a
consequence the  jdoConnectionURL and related properties for the mysql
metastore are loaded allowing the metadata to be stored in mysql

2012/12/9 Stephen Boesch <[EMAIL PROTECTED]>

> The first element of the classpath is the right one already.. but I STILL
> get the hive-site.xml is not found in classpath.  Only hive gives me
> issues.  hdfs, mapred, hbase are all running fine.
>
> HADOOP_CLASSPATH=:*/shared/hive/conf:*
> /shared/hive/lib/antlr-runtime-3.0.1.jar:/shared/hive/lib/commons-cli-1.2.jar:/shared/hive/lib/commons-codec-1.3.jar:/shared/hive/lib/commons-collections-3.2.1.jar:/shared/hive/lib/commons-dbcp-1.4.jar:/shared/hive/lib/commons-lang-2.4.jar:/shared/hive/lib/commons-logging-1.0.4.jar:/shared/hive/lib/commons-logging-api-1.0.4.jar:/shared/hive/lib/commons-pool-1.5.4.jar:/shared/hive/lib/datanucleus-connectionpool-2.0.3.jar:/shared/hive/lib/datanucleus-core-2.0.3.jar:/shared/hive/lib/datanucleus-enhancer-2.0.3.jar:/shared/hive/lib/datanucleus-rdbms-2.0.3.jar:/shared/hive/lib/derby-10.4.2.0.jar:/shared/hive/lib/guava-r09.jar:/shared/hive/lib/hbase-0.92.0.jar:/shared/hive/lib/hbase-0.92.0-tests.jar:/shared/hive/lib/hive-builtins-0.9.0.jar:/shared/hive/lib/hive-cli-0.9.0.jar:/shared/hive/lib/hive-common-0.9.0.jar:/shared/hive/lib/hive-contrib-0.9.0.jar:/shared/hive/lib/hive_contrib.jar:/shared/hive/lib/hive-exec-0.9.0.jar:/shared/hive/lib/hive-hbase-handler-0.9.0.jar:/shared/hive/lib/hive-hwi-0.9.0.jar:/shared/hive/lib/hive-jdbc-0.9.0.jar:/shared/hive/lib/hive-metastore-0.9.0.jar:/shared/hive/lib/hive-pdk-0.9.0.jar:/shared/hive/lib/hive-serde-0.9.0.jar:/shared/hive/lib/hive-service-0.9.0.jar:/shared/hive/lib/hive-shims-0.9.0.jar:/shared/hive/lib/jackson-core-asl-1.8.8.jar:/shared/hive/lib/jackson-jaxrs-1.8.8.jar:/shared/hive/lib/jackson-mapper-asl-1.8.8.jar:/shared/hive/lib/jackson-xc-1.8.8.jar:/shared/hive/lib/JavaEWAH-0.3.2.jar:/shared/hive/lib/jdo2-api-2.3-ec.jar:/shared/hive/lib/jline-0.9.94.
>
> steve@mithril:/shared/hive/bin$ *ls -lrta /shared/hive/conf/hive-site.xml*
> -rw-r--r-- 1 steve steve 4415 2012-11-25 23:02
> /shared/hive/conf/hive-site.xml
>
>
>
> 2012/11/30 Lauren Yang <[EMAIL PROTECTED]>
>
>>  You can see if the classpath is being passed correctly to hadoop by
>> putting in an echo statement around line 150 of the hive cli script where
>> it passes the CLASSPATH variable to HADOOP_CLASSPATH.****
>>
>> # pass classpath to hadoop****
>>
>> export HADOOP_CLASSPATH="${HADOOP_CLASSPATH}:${CLASSPATH}"****
>>
>> ** **
>>
>> You could also echo the classpath in the hadoop script (in your
>> HADOOP_HOME\bin directory) to see if the classpath is being passed
>> correctly to the time when the cli jar is invoked.****
>>
>> ** **
>>
>> As far as the logs location, if this is not set in your hive-site.xml,
>> you can set it by passing  in HIVE_OPTS when you invoke the command line.
>> ****
>>
>> ** **
>>
>> Like so:****
>>
>> EXPORT HIVE_OPTS= -hiveconf hive.log.dir=$ HIVE_HOME\logs"****
>>
>> Then run “hive”****
>>
>> ** **
>>
>> Or:****
>>
>> Run “hive --hiveconf hive.log.dir=$ HIVE_HOME\logs”****
>>
>> ** **
>>
>> Thanks,****
>>
>> Lauren****
>>
>> ** **
>>
>> ** **
>>
>> *From:* Stephen Boesch [mailto:[EMAIL PROTECTED]]
>> *Sent:* Friday, November 30, 2012 12:16 AM
>> *To:* [EMAIL PROTECTED]
>> *Subject:* Re: hive-site.xml not found on classpath****
>>
>> ** **
>>
>> running 0.9.0 (you can see it from the classpath shown below);****
>>
>> ** **
>>
>> steve@mithril:/shared/cdh4$ echo $HIVE_CONF_DIR****
>>
>> /shared/hive/conf****