Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Re: hive-site.xml not found on classpath


Copy link to this message
-
RE: hive-site.xml not found on classpath
You can see if the classpath is being passed correctly to hadoop by putting in an echo statement around line 150 of the hive cli script where it passes the CLASSPATH variable to HADOOP_CLASSPATH.
# pass classpath to hadoop
export HADOOP_CLASSPATH="${HADOOP_CLASSPATH}:${CLASSPATH}"

You could also echo the classpath in the hadoop script (in your HADOOP_HOME\bin directory) to see if the classpath is being passed correctly to the time when the cli jar is invoked.

As far as the logs location, if this is not set in your hive-site.xml, you can set it by passing  in HIVE_OPTS when you invoke the command line.

Like so:
EXPORT HIVE_OPTS= -hiveconf hive.log.dir=$ HIVE_HOME\logs"
Then run "hive"

Or:
Run "hive --hiveconf hive.log.dir=$ HIVE_HOME\logs"

Thanks,
Lauren
From: Stephen Boesch [mailto:[EMAIL PROTECTED]]
Sent: Friday, November 30, 2012 12:16 AM
To: [EMAIL PROTECTED]
Subject: Re: hive-site.xml not found on classpath

running 0.9.0 (you can see it from the classpath shown below);

steve@mithril:/shared/cdh4$ echo $HIVE_CONF_DIR
/shared/hive/conf
steve@mithril:/shared/cdh4$ ls -l $HIVE_CONF_DIR
total 152
-rw-r--r-- 1 steve steve 46053 2011-12-13 00:36 hive-default.xml.template
-rw-r--r-- 1 steve steve  1615 2012-11-13 23:37 hive-env.bullshit.sh<http://hive-env.bullshit.sh>
-rw-r--r-- 1 steve steve  1671 2012-11-28 01:43 hive-env.sh
-rw-r--r-- 1 steve steve  1593 2011-12-13 00:36 hive-env.sh.template
-rw-r--r-- 1 steve steve  1637 2011-12-13 00:36 hive-exec-log4j.properties.template
-rw-r--r-- 1 root  root   2056 2012-11-28 01:38 hive-log4j.properties
-rw-r--r-- 1 steve steve  2056 2012-03-25 12:49 hive-log4j.properties.template
-rw-r--r-- 1 steve steve  4415 2012-11-25 23:02 hive-site.xml
steve@mithril:/shared/cdh4$ echo $HIVE_HOME
/shared/hive
steve@mithril:/shared/cdh4$ echo $(which hive)
/shared/hive/bin/hive

also you can see the hive/conf is the first entry

After adding the debug statement:

classpath=/shared/hive/conf:/shared/hive/lib/antlr-runtime-3.0.1.jar:/shared/hive/lib/commons-cli-1.2.jar:/shared/hive/lib/commons-codec-1.3.jar:/shared/hive/lib/commons-collections-3.2.1.jar:/shared/hive/lib/commons-dbcp-1.4.jar:/shared/hive/lib/commons-lang-2.4.jar:/shared/hive/lib/commons-logging-1.0.4.jar:/shared/hive/lib/commons-logging-api-1.0.4.jar:/shared/hive/lib/commons-pool-1.5.4.jar:/shared/hive/lib/datanucleus-connectionpool-2.0.3.jar:/shared/hive/lib/datanucleus-core-2.0.3.jar:/shared/hive/lib/datanucleus-enhancer-2.0.3.jar:/shared/hive/lib/datanucleus-rdbms-2.0.3.jar:/shared/hive/lib/derby-10.4.2.0.jar:/shared/hive/lib/guava-r09.jar:/shared/hive/lib/hbase-0.92.0.jar:/shared/hive/lib/hbase-0.92.0-tests.jar:/shared/hive/lib/hive-builtins-0.9.0.jar:/shared/hive/lib/hive-cli-0.9.0.jar:/shared/hive/lib/hive-common-0.9.0.jar:/shared/hive/lib/hive-contrib-0.9.0.jar:/shared/hive/lib/hive_contrib.jar:/shared/hive/lib/hive-exec-0.9.0.jar:/shared/hive/lib/hive-hbase-handler-0.9.0.jar:/shared/hive/lib/hive-hwi-0.9.0.jar:/shared/hive/lib/hive-jdbc-0.9.0.jar:/shared/hive/lib/hive-metastore-0.9.0.jar:/shared/hive/lib/hive-pdk-0.9.0.jar:/shared/hive/lib/hive-serde-0.9.0.jar:/shared/hive/lib/hive-service-0.9.0.jar:/shared/hive/lib/hive-shims-0.9.0.jar:/shared/hive/lib/jackson-core-asl-1.8.8.jar:/shared/hive/lib/jackson-jaxrs-1.8.8.jar:/shared/hive/lib/jackson-mapper-asl-1.8.8.jar:/shared/hive/lib/jackson-xc-1.8.8.jar:/shared/hive/lib/JavaEWAH-0.3.2.jar:/shared/hive/lib/jdo2-api-2.3-ec.jar:/shared/hive/lib/jline-0.9.94.jar:/shared/hive/lib/json-20090211.jar:/shared/hive/lib/libfb303-0.7.0.jar:/shared/hive/lib/libfb303.jar:/shared/hive/lib/libthrift-0.7.0.jar:/shared/hive/lib/libthrift.jar:/shared/hive/lib/log4j-1.2.16.jar:/shared/hive/lib/mysql-connector-java-5.1.18-bin.jar:/shared/hive/lib/slf4j-api-1.6.1.jar:/shared/hive/lib/slf4j-log4j12-1.6.1.jar:/shared/hive/lib/stringtemplate-3.1-b1.jar:/shared/hive/lib/zookeeper-3.4.3.jar:
But even so:

  *   the log dir is still wrong (writing to /tmp/${user}/hive.log instead of $HIVE_HOME/logs)
  *   the following message in the log file
2012-11-30 00:12:31,775 WARN  conf.HiveConf (HiveConf.java:<clinit>(70)) - hive-site.xml not found on CLASSPATH

2012/11/30 Bing Li <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
which version of hive do you use?

Could you try to add the following debug line in bin/hive before hive real executes, and see the result?

echo "CLASSPATH=$CLASSPATH"

if [ "$TORUN" = "" ]; then
   echo "Service $SERVICE not found"
   echo "Available Services: $SERVICE_LIST"
   exit 7
else
   $TORUN "$@"
fi

The version I used is 0.9.0
2012/11/30 Stephen Boesch <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Yes i do mean the log is in the wrong location, since it was set to a persistent path in the $HIVE_CONF_DIR/lhive-log4j.properties.

None of the files in that directory appear to be picked up properly: neither the hive-site.xml nor log4j.properties.

I have put echo statements into the 'hive" and the hive-config.sh  shell scripts and the echo statements prove that  HIVE_CONF_DIR is set properly:  /shared/hive/conf

But even so the following problems occur:

  *   the message "hive-site.xml is not found in the classpath"
  *   none of the hive-site.xml values are taking properly
  *   the log4j.properties in that same directory is not taking effect.
2012/11/29 Bing Li <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Hi, Stephen
what did you mean the "wrong place under /tmp" in
"I am seeing the following message in the logs (which are in the wrong place under /tmp..)" ?

Did you mean that you set a different log dir but it didn't work?

the log dir should be set in conf/hive-log4j.properties, conf/hive-exec-log4j.properties
and you can try to reset HIVE_CONF_DIR in conf/hive-env.sh with 'export" command.

- Bing

2012/11/30 Stephen Boesch <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
thought i mentioned in the posts those were already set and verified.. but yes in any case that's first thing looked at.

steve@mithri
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB