Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive >> mail # user >> Re: hive-site.xml not found on classpath


+
Stephen Boesch 2012-11-29, 08:56
+
Shreepadma Venugopalan 2012-11-29, 20:42
+
Stephen Boesch 2012-11-29, 20:46
+
kulkarni.swarnim@...) 2012-11-29, 20:54
+
Stephen Boesch 2012-11-29, 21:02
+
Bing Li 2012-11-30, 07:01
+
Stephen Boesch 2012-11-30, 07:05
+
Bing Li 2012-11-30, 08:01
+
Stephen Boesch 2012-11-30, 08:15
+
Lauren Yang 2012-11-30, 18:46
+
Stephen Boesch 2012-12-09, 08:06
Copy link to this message
-
Re: hive-site.xml not found on classpath
I ended up patching the HiveConf.java  . If  hive-site.xml were not found
on the classpath then:

   - an   o.a.h.fs.Path object is created from
    System.getenv("HIVE_CONF_DIR") + File.seperator + "hive-site.xml"
   - the Path is sent to the base class Configuration.addResource - whiich
   btw accepts either resourceURL's or Path objects

the problem was resolved  with that change to HiveConf.java:  the
hive-site.xml is loaded properly via the Path (instead of URL)  and as a
consequence the  jdoConnectionURL and related properties for the mysql
metastore are loaded allowing the metadata to be stored in mysql

2012/12/9 Stephen Boesch <[EMAIL PROTECTED]>

> The first element of the classpath is the right one already.. but I STILL
> get the hive-site.xml is not found in classpath.  Only hive gives me
> issues.  hdfs, mapred, hbase are all running fine.
>
> HADOOP_CLASSPATH=:*/shared/hive/conf:*
> /shared/hive/lib/antlr-runtime-3.0.1.jar:/shared/hive/lib/commons-cli-1.2.jar:/shared/hive/lib/commons-codec-1.3.jar:/shared/hive/lib/commons-collections-3.2.1.jar:/shared/hive/lib/commons-dbcp-1.4.jar:/shared/hive/lib/commons-lang-2.4.jar:/shared/hive/lib/commons-logging-1.0.4.jar:/shared/hive/lib/commons-logging-api-1.0.4.jar:/shared/hive/lib/commons-pool-1.5.4.jar:/shared/hive/lib/datanucleus-connectionpool-2.0.3.jar:/shared/hive/lib/datanucleus-core-2.0.3.jar:/shared/hive/lib/datanucleus-enhancer-2.0.3.jar:/shared/hive/lib/datanucleus-rdbms-2.0.3.jar:/shared/hive/lib/derby-10.4.2.0.jar:/shared/hive/lib/guava-r09.jar:/shared/hive/lib/hbase-0.92.0.jar:/shared/hive/lib/hbase-0.92.0-tests.jar:/shared/hive/lib/hive-builtins-0.9.0.jar:/shared/hive/lib/hive-cli-0.9.0.jar:/shared/hive/lib/hive-common-0.9.0.jar:/shared/hive/lib/hive-contrib-0.9.0.jar:/shared/hive/lib/hive_contrib.jar:/shared/hive/lib/hive-exec-0.9.0.jar:/shared/hive/lib/hive-hbase-handler-0.9.0.jar:/shared/hive/lib/hive-hwi-0.9.0.jar:/shared/hive/lib/hive-jdbc-0.9.0.jar:/shared/hive/lib/hive-metastore-0.9.0.jar:/shared/hive/lib/hive-pdk-0.9.0.jar:/shared/hive/lib/hive-serde-0.9.0.jar:/shared/hive/lib/hive-service-0.9.0.jar:/shared/hive/lib/hive-shims-0.9.0.jar:/shared/hive/lib/jackson-core-asl-1.8.8.jar:/shared/hive/lib/jackson-jaxrs-1.8.8.jar:/shared/hive/lib/jackson-mapper-asl-1.8.8.jar:/shared/hive/lib/jackson-xc-1.8.8.jar:/shared/hive/lib/JavaEWAH-0.3.2.jar:/shared/hive/lib/jdo2-api-2.3-ec.jar:/shared/hive/lib/jline-0.9.94.
>
> steve@mithril:/shared/hive/bin$ *ls -lrta /shared/hive/conf/hive-site.xml*
> -rw-r--r-- 1 steve steve 4415 2012-11-25 23:02
> /shared/hive/conf/hive-site.xml
>
>
>
> 2012/11/30 Lauren Yang <[EMAIL PROTECTED]>
>
>>  You can see if the classpath is being passed correctly to hadoop by
>> putting in an echo statement around line 150 of the hive cli script where
>> it passes the CLASSPATH variable to HADOOP_CLASSPATH.****
>>
>> # pass classpath to hadoop****
>>
>> export HADOOP_CLASSPATH="${HADOOP_CLASSPATH}:${CLASSPATH}"****
>>
>> ** **
>>
>> You could also echo the classpath in the hadoop script (in your
>> HADOOP_HOME\bin directory) to see if the classpath is being passed
>> correctly to the time when the cli jar is invoked.****
>>
>> ** **
>>
>> As far as the logs location, if this is not set in your hive-site.xml,
>> you can set it by passing  in HIVE_OPTS when you invoke the command line.
>> ****
>>
>> ** **
>>
>> Like so:****
>>
>> EXPORT HIVE_OPTS= -hiveconf hive.log.dir=$ HIVE_HOME\logs"****
>>
>> Then run “hive”****
>>
>> ** **
>>
>> Or:****
>>
>> Run “hive --hiveconf hive.log.dir=$ HIVE_HOME\logs”****
>>
>> ** **
>>
>> Thanks,****
>>
>> Lauren****
>>
>> ** **
>>
>> ** **
>>
>> *From:* Stephen Boesch [mailto:[EMAIL PROTECTED]]
>> *Sent:* Friday, November 30, 2012 12:16 AM
>> *To:* [EMAIL PROTECTED]
>> *Subject:* Re: hive-site.xml not found on classpath****
>>
>> ** **
>>
>> running 0.9.0 (you can see it from the classpath shown below);****
>>
>> ** **
>>
>> steve@mithril:/shared/cdh4$ echo $HIVE_CONF_DIR****
>>
>> /shared/hive/conf****
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB