i am running under user steve.  the latest log (where this shows up ) is
 /tmp/steve/hive.log
2012/11/29 Viral Bajaria <[EMAIL PROTECTED]>

> You are seeing this error when you run the hive cli or in the tasktracker
> logs when you run a query ?
>
> On Thu, Nov 29, 2012 at 12:42 AM, Stephen Boesch <[EMAIL PROTECTED]>wrote:
>
>>
>> I am seeing the following message in the logs (which are in the wrong
>> place under /tmp..)
>>
>> hive-site.xml not found on classpath
>>
>> My hive-site.xml is under the standard location  $HIVE_HOME/conf so this
>> should not happen.
>>
>> Now some posts have talked that the HADOOP_CLASSPATH was mangled.  Mine
>> is not..
>>
>> So what is the underlying issue here?
>>
>> Thanks
>>
>> stephenb
>>
>
>

Are you seeing this message when your bring up the standalone hive cli by
running 'hive'?
On Thu, Nov 29, 2012 at 12:56 AM, Stephen Boesch <[EMAIL PROTECTED]> wrote:

> i am running under user steve.  the latest log (where this shows up ) is
>  /tmp/steve/hive.log
>
>
> 2012/11/29 Viral Bajaria <[EMAIL PROTECTED]>
>
>> You are seeing this error when you run the hive cli or in the tasktracker
>> logs when you run a query ?
>>
>> On Thu, Nov 29, 2012 at 12:42 AM, Stephen Boesch <[EMAIL PROTECTED]>wrote:
>>
>>>
>>> I am seeing the following message in the logs (which are in the wrong
>>> place under /tmp..)
>>>
>>>  hive-site.xml not found on classpath
>>>
>>> My hive-site.xml is under the standard location  $HIVE_HOME/conf so this
>>> should not happen.
>>>
>>> Now some posts have talked that the HADOOP_CLASSPATH was mangled.  Mine
>>> is not..
>>>
>>> So what is the underlying issue here?
>>>
>>> Thanks
>>>
>>> stephenb
>>>
>>
>>
>

Yes.
2012/11/29 Shreepadma Venugopalan <[EMAIL PROTECTED]>

> Are you seeing this message when your bring up the standalone hive cli by
> running 'hive'?
>
>
> On Thu, Nov 29, 2012 at 12:56 AM, Stephen Boesch <[EMAIL PROTECTED]>wrote:
>
>> i am running under user steve.  the latest log (where this shows up ) is
>>  /tmp/steve/hive.log
>>
>>
>> 2012/11/29 Viral Bajaria <[EMAIL PROTECTED]>
>>
>>> You are seeing this error when you run the hive cli or in the
>>> tasktracker logs when you run a query ?
>>>
>>> On Thu, Nov 29, 2012 at 12:42 AM, Stephen Boesch <[EMAIL PROTECTED]>wrote:
>>>
>>>>
>>>> I am seeing the following message in the logs (which are in the wrong
>>>> place under /tmp..)
>>>>
>>>>  hive-site.xml not found on classpath
>>>>
>>>> My hive-site.xml is under the standard location  $HIVE_HOME/conf so
>>>> this should not happen.
>>>>
>>>> Now some posts have talked that the HADOOP_CLASSPATH was mangled.  Mine
>>>> is not..
>>>>
>>>> So what is the underlying issue here?
>>>>
>>>> Thanks
>>>>
>>>> stephenb
>>>>
>>>
>>>
>>
>

Have you tried setting HIVE_HOME and HIVE_CONF_DIR?
On Thu, Nov 29, 2012 at 2:46 PM, Stephen Boesch <[EMAIL PROTECTED]> wrote:

> Yes.
>
>
> 2012/11/29 Shreepadma Venugopalan <[EMAIL PROTECTED]>
>
>> Are you seeing this message when your bring up the standalone hive cli by
>> running 'hive'?
>>
>>
>> On Thu, Nov 29, 2012 at 12:56 AM, Stephen Boesch <[EMAIL PROTECTED]>wrote:
>>
>>> i am running under user steve.  the latest log (where this shows up ) is
>>>  /tmp/steve/hive.log
>>>
>>>
>>> 2012/11/29 Viral Bajaria <[EMAIL PROTECTED]>
>>>
>>>> You are seeing this error when you run the hive cli or in the
>>>> tasktracker logs when you run a query ?
>>>>
>>>> On Thu, Nov 29, 2012 at 12:42 AM, Stephen Boesch <[EMAIL PROTECTED]>wrote:
>>>>
>>>>>
>>>>> I am seeing the following message in the logs (which are in the wrong
>>>>> place under /tmp..)
>>>>>
>>>>>  hive-site.xml not found on classpath
>>>>>
>>>>> My hive-site.xml is under the standard location  $HIVE_HOME/conf so
>>>>> this should not happen.
>>>>>
>>>>> Now some posts have talked that the HADOOP_CLASSPATH was mangled.
>>>>>  Mine is not..
>>>>>
>>>>> So what is the underlying issue here?
>>>>>
>>>>> Thanks
>>>>>
>>>>> stephenb
>>>>>
>>>>
>>>>
>>>
>>
>
--
Swarnim

thought i mentioned in the posts those were already set and verified.. but
yes in any case that's first thing looked at.

steve@mithril:~$ echo $HIVE_CONF_DIR
/shared/hive/conf
steve@mithril:~$ echo $HIVE_HOME
/shared/hive
2012/11/29 [EMAIL PROTECTED] <[EMAIL PROTECTED]>

> Have you tried setting HIVE_HOME and HIVE_CONF_DIR?
>
>
> On Thu, Nov 29, 2012 at 2:46 PM, Stephen Boesch <[EMAIL PROTECTED]> wrote:
>
>> Yes.
>>
>>
>> 2012/11/29 Shreepadma Venugopalan <[EMAIL PROTECTED]>
>>
>>> Are you seeing this message when your bring up the standalone hive cli
>>> by running 'hive'?
>>>
>>>
>>> On Thu, Nov 29, 2012 at 12:56 AM, Stephen Boesch <[EMAIL PROTECTED]>wrote:
>>>
>>>> i am running under user steve.  the latest log (where this shows up )
>>>> is  /tmp/steve/hive.log
>>>>
>>>>
>>>> 2012/11/29 Viral Bajaria <[EMAIL PROTECTED]>
>>>>
>>>>> You are seeing this error when you run the hive cli or in the
>>>>> tasktracker logs when you run a query ?
>>>>>
>>>>> On Thu, Nov 29, 2012 at 12:42 AM, Stephen Boesch <[EMAIL PROTECTED]>wrote:
>>>>>
>>>>>>
>>>>>> I am seeing the following message in the logs (which are in the wrong
>>>>>> place under /tmp..)
>>>>>>
>>>>>>  hive-site.xml not found on classpath
>>>>>>
>>>>>> My hive-site.xml is under the standard location  $HIVE_HOME/conf so
>>>>>> this should not happen.
>>>>>>
>>>>>> Now some posts have talked that the HADOOP_CLASSPATH was mangled.
>>>>>>  Mine is not..
>>>>>>
>>>>>> So what is the underlying issue here?
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>> stephenb
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
>
> --
> Swarnim
>

Hi, Stephen
what did you mean the "wrong place under /tmp" in
"I am seeing the following message in the logs (which are in the wrong
place under /tmp..)" ?

Did you mean that you set a different log dir but it didn't work?

the log dir should be set in conf/hive-log4j.properties,
conf/hive-exec-log4j.properties
and you can try to reset HIVE_CONF_DIR in conf/hive-env.sh with ‘export"
command.

- Bing

2012/11/30 Stephen Boesch <[EMAIL PROTECTED]>

> thought i mentioned in the posts those were already set and verified.. but
> yes in any case that's first thing looked at.
>
> steve@mithril:~$ echo $HIVE_CONF_DIR
> /shared/hive/conf
> steve@mithril:~$ echo $HIVE_HOME
> /shared/hive
>
>
> 2012/11/29 [EMAIL PROTECTED] <[EMAIL PROTECTED]>
>
> Have you tried setting HIVE_HOME and HIVE_CONF_DIR?
>>
>>
>> On Thu, Nov 29, 2012 at 2:46 PM, Stephen Boesch <[EMAIL PROTECTED]>wrote:
>>
>>> Yes.
>>>
>>>
>>> 2012/11/29 Shreepadma Venugopalan <[EMAIL PROTECTED]>
>>>
>>>> Are you seeing this message when your bring up the standalone hive cli
>>>> by running 'hive'?
>>>>
>>>>
>>>> On Thu, Nov 29, 2012 at 12:56 AM, Stephen Boesch <[EMAIL PROTECTED]>wrote:
>>>>
>>>>> i am running under user steve.  the latest log (where this shows up )
>>>>> is  /tmp/steve/hive.log
>>>>>
>>>>>
>>>>> 2012/11/29 Viral Bajaria <[EMAIL PROTECTED]>
>>>>>
>>>>>> You are seeing this error when you run the hive cli or in the
>>>>>> tasktracker logs when you run a query ?
>>>>>>
>>>>>> On Thu, Nov 29, 2012 at 12:42 AM, Stephen Boesch <[EMAIL PROTECTED]>wrote:
>>>>>>
>>>>>>>
>>>>>>> I am seeing the following message in the logs (which are in the
>>>>>>> wrong place under /tmp..)
>>>>>>>
>>>>>>>  hive-site.xml not found on classpath
>>>>>>>
>>>>>>> My hive-site.xml is under the standard location  $HIVE_HOME/conf so
>>>>>>> this should not happen.
>>>>>>>
>>>>>>> Now some posts have talked that the HADOOP_CLASSPATH was mangled.
>>>>>>>  Mine is not..
>>>>>>>
>>>>>>> So what is the underlying issue here?
>>>>>>>
>>>>>>> Thanks
>>>>>>>
>>>>>>> stephenb
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>>
>> --
>> Swarnim
>>
>
>

Yes i do mean the log is in the wrong location, since it was set to a
persistent path in the $HIVE_CONF_DIR/lhive-log4j.properties.

None of the files in that directory appear to be picked up properly:
neither the hive-site.xml nor log4j.properties.

I have put echo statements into the 'hive" and the hive-config.sh  shell
scripts and the echo statements prove that  HIVE_CONF_DIR is set properly:
 /shared/hive/conf

But even so the following problems occur:

   - the message "hive-site.xml is not found in the classpath"
   - none of the hive-site.xml values are taking properly
   - the log4j.properties in that same directory is not taking effect.
2012/11/29 Bing Li <[EMAIL PROTECTED]>

> Hi, Stephen
> what did you mean the "wrong place under /tmp" in
> "I am seeing the following message in the logs (which are in the wrong
> place under /tmp..)" ?
>
> Did you mean that you set a different log dir but it didn't work?
>
> the log dir should be set in conf/hive-log4j.properties,
> conf/hive-exec-log4j.properties
> and you can try to reset HIVE_CONF_DIR in conf/hive-env.sh with ‘export"
> command.
>
> - Bing
>
>
> 2012/11/30 Stephen Boesch <[EMAIL PROTECTED]>
>
>> thought i mentioned in the posts those were already set and verified..
>> but yes in any case that's first thing looked at.
>>
>> steve@mithril:~$ echo $HIVE_CONF_DIR
>> /shared/hive/conf
>> steve@mithril:~$ echo $HIVE_HOME
>> /shared/hive
>>
>>
>> 2012/11/29 [EMAIL PROTECTED] <[EMAIL PROTECTED]>
>>
>> Have you tried setting HIVE_HOME and HIVE_CONF_DIR?
>>>
>>>
>>> On Thu, Nov 29, 2012 at 2:46 PM, Stephen Boesch <[EMAIL PROTECTED]>wrote:
>>>
>>>> Yes.
>>>>
>>>>
>>>> 2012/11/29 Shreepadma Venugopalan <[EMAIL PROTECTED]>
>>>>
>>>>> Are you seeing this message when your bring up the standalone hive cli
>>>>> by running 'hive'?
>>>>>
>>>>>
>>>>> On Thu, Nov 29, 2012 at 12:56 AM, Stephen Boesch <[EMAIL PROTECTED]>wrote:
>>>>>
>>>>>> i am running under user steve.  the latest log (where this shows up )
>>>>>> is  /tmp/steve/hive.log
>>>>>>
>>>>>>
>>>>>> 2012/11/29 Viral Bajaria <[EMAIL PROTECTED]>
>>>>>>
>>>>>>> You are seeing this error when you run the hive cli or in the
>>>>>>> tasktracker logs when you run a query ?
>>>>>>>
>>>>>>> On Thu, Nov 29, 2012 at 12:42 AM, Stephen Boesch <[EMAIL PROTECTED]>wrote:
>>>>>>>
>>>>>>>>
>>>>>>>> I am seeing the following message in the logs (which are in the
>>>>>>>> wrong place under /tmp..)
>>>>>>>>
>>>>>>>>  hive-site.xml not found on classpath
>>>>>>>>
>>>>>>>> My hive-site.xml is under the standard location  $HIVE_HOME/conf so
>>>>>>>> this should not happen.
>>>>>>>>
>>>>>>>> Now some posts have talked that the HADOOP_CLASSPATH was mangled.
>>>>>>>>  Mine is not..
>>>>>>>>
>>>>>>>> So what is the underlying issue here?
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>>
>>>>>>>> stephenb
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>> Swarnim
>>>
>>
>>
>

which version of hive do you use?

Could you try to add the following debug line in bin/hive before hive real
executes, and see the result?

*echo "CLASSPATH=$CLASSPATH"*

if [ "$TORUN" = "" ]; then
   echo "Service $SERVICE not found"
   echo "Available Services: $SERVICE_LIST"
   exit 7
else
   $TORUN "$@"
fi

The version I used is 0.9.0
2012/11/30 Stephen Boesch <[EMAIL PROTECTED]>

> Yes i do mean the log is in the wrong location, since it was set to a
> persistent path in the $HIVE_CONF_DIR/lhive-log4j.properties.
>
> None of the files in that directory appear to be picked up properly:
> neither the hive-site.xml nor log4j.properties.
>
> I have put echo statements into the 'hive" and the hive-config.sh  shell
> scripts and the echo statements prove that  HIVE_CONF_DIR is set properly:
>  /shared/hive/conf
>
> But even so the following problems occur:
>
>    - the message "hive-site.xml is not found in the classpath"
>    - none of the hive-site.xml values are taking properly
>    - the log4j.properties in that same directory is not taking effect.
>
>
>
>
> 2012/11/29 Bing Li <[EMAIL PROTECTED]>
>
>> Hi, Stephen
>> what did you mean the "wrong place under /tmp" in
>> "I am seeing the following message in the logs (which are in the wrong
>> place under /tmp..)" ?
>>
>> Did you mean that you set a different log dir but it didn't work?
>>
>> the log dir should be set in conf/hive-log4j.properties,
>> conf/hive-exec-log4j.properties
>> and you can try to reset HIVE_CONF_DIR in conf/hive-env.sh with ‘export"
>> command.
>>
>> - Bing
>>
>>
>> 2012/11/30 Stephen Boesch <[EMAIL PROTECTED]>
>>
>>> thought i mentioned in the posts those were already set and verified..
>>> but yes in any case that's first thing looked at.
>>>
>>> steve@mithril:~$ echo $HIVE_CONF_DIR
>>> /shared/hive/conf
>>> steve@mithril:~$ echo $HIVE_HOME
>>> /shared/hive
>>>
>>>
>>> 2012/11/29 [EMAIL PROTECTED] <[EMAIL PROTECTED]>
>>>
>>> Have you tried setting HIVE_HOME and HIVE_CONF_DIR?
>>>>
>>>>
>>>> On Thu, Nov 29, 2012 at 2:46 PM, Stephen Boesch <[EMAIL PROTECTED]>wrote:
>>>>
>>>>> Yes.
>>>>>
>>>>>
>>>>> 2012/11/29 Shreepadma Venugopalan <[EMAIL PROTECTED]>
>>>>>
>>>>>> Are you seeing this message when your bring up the standalone hive
>>>>>> cli by running 'hive'?
>>>>>>
>>>>>>
>>>>>> On Thu, Nov 29, 2012 at 12:56 AM, Stephen Boesch <[EMAIL PROTECTED]>wrote:
>>>>>>
>>>>>>> i am running under user steve.  the latest log (where this shows up
>>>>>>> ) is  /tmp/steve/hive.log
>>>>>>>
>>>>>>>
>>>>>>> 2012/11/29 Viral Bajaria <[EMAIL PROTECTED]>
>>>>>>>
>>>>>>>> You are seeing this error when you run the hive cli or in the
>>>>>>>> tasktracker logs when you run a query ?
>>>>>>>>
>>>>>>>> On Thu, Nov 29, 2012 at 12:42 AM, Stephen Boesch <[EMAIL PROTECTED]
>>>>>>>> > wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>> I am seeing the following message in the logs (which are in the
>>>>>>>>> wrong place under /tmp..)
>>>>>>>>>
>>>>>>>>>  hive-site.xml not found on classpath
>>>>>>>>>
>>>>>>>>> My hive-site.xml is under the standard location  $HIVE_HOME/conf
>>>>>>>>> so this should not happen.
>>>>>>>>>
>>>>>>>>> Now some posts have talked that the HADOOP_CLASSPATH was mangled.
>>>>>>>>>  Mine is not..
>>>>>>>>>
>>>>>>>>> So what is the underlying issue here?
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>>
>>>>>>>>> stephenb
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Swarnim
>>>>
>>>
>>>
>>
>

running 0.9.0 (you can see it from the classpath shown below);

steve@mithril:/shared/cdh4$ echo $HIVE_CONF_DIR
/shared/hive/conf
steve@mithril:/shared/cdh4$ ls -l $HIVE_CONF_DIR
total 152
-rw-r--r-- 1 steve steve 46053 2011-12-13 00:36 hive-default.xml.template
-rw-r--r-- 1 steve steve  1615 2012-11-13 23:37 hive-env.bullshit.sh
-rw-r--r-- 1 steve steve  1671 2012-11-28 01:43 hive-env.sh
-rw-r--r-- 1 steve steve  1593 2011-12-13 00:36 hive-env.sh.template
-rw-r--r-- 1 steve steve  1637 2011-12-13 00:36
hive-exec-log4j.properties.template
-rw-r--r-- 1 root  root   2056 2012-11-28 01:38 hive-log4j.properties
-rw-r--r-- 1 steve steve  2056 2012-03-25 12:49
hive-log4j.properties.template
-rw-r--r-- 1 steve steve  4415 2012-11-25 23:02 hive-site.xml
steve@mithril:/shared/cdh4$ echo $HIVE_HOME
/shared/hive
steve@mithril:/shared/cdh4$ echo $(which hive)
/shared/hive/bin/hive

also you can see the hive/conf is the first entry

After adding the debug statement:

classpath=*/shared/hive/conf:*
/shared/hive/lib/antlr-runtime-3.0.1.jar:/shared/hive/lib/commons-cli-1.2.jar:/shared/hive/lib/commons-codec-1.3.jar:/shared/hive/lib/commons-collections-3.2.1.jar:/shared/hive/lib/commons-dbcp-1.4.jar:/shared/hive/lib/commons-lang-2.4.jar:/shared/hive/lib/commons-logging-1.0.4.jar:/shared/hive/lib/commons-logging-api-1.0.4.jar:/shared/hive/lib/commons-pool-1.5.4.jar:/shared/hive/lib/datanucleus-connectionpool-2.0.3.jar:/shared/hive/lib/datanucleus-core-2.0.3.jar:/shared/hive/lib/datanucleus-enhancer-2.0.3.jar:/shared/hive/lib/datanucleus-rdbms-2.0.3.jar:/shared/hive/lib/derby-10.4.2.0.jar:/shared/hive/lib/guava-r09.jar:/shared/hive/lib/hbase-0.92.0.jar:/shared/hive/lib/hbase-0.92.0-tests.jar:/shared/hive/lib/hive-builtins-0.9.0.jar:/shared/hive/lib/hive-cli-0.9.0.jar:/shared/hive/lib/hive-common-0.9.0.jar:/shared/hive/lib/hive-contrib-0.9.0.jar:/shared/hive/lib/hive_contrib.jar:/shared/hive/lib/hive-exec-0.9.0.jar:/shared/hive/lib/hive-hbase-handler-0.9.0.jar:/shared/hive/lib/hive-hwi-0.9.0.jar:/shared/hive/lib/hive-jdbc-0.9.0.jar:/shared/hive/lib/hive-metastore-0.9.0.jar:/shared/hive/lib/hive-pdk-0.9.0.jar:/shared/hive/lib/hive-serde-0.9.0.jar:/shared/hive/lib/hive-service-0.9.0.jar:/shared/hive/lib/hive-shims-0.9.0.jar:/shared/hive/lib/jackson-core-asl-1.8.8.jar:/shared/hive/lib/jackson-jaxrs-1.8.8.jar:/shared/hive/lib/jackson-mapper-asl-1.8.8.jar:/shared/hive/lib/jackson-xc-1.8.8.jar:/shared/hive/lib/JavaEWAH-0.3.2.jar:/shared/hive/lib/jdo2-api-2.3-ec.jar:/shared/hive/lib/jline-0.9.94.jar:/shared/hive/lib/json-20090211.jar:/shared/hive/lib/libfb303-0.7.0.jar:/shared/hive/lib/libfb303.jar:/shared/hive/lib/libthrift-0.7.0.jar:/shared/hive/lib/libthrift.jar:/shared/hive/lib/log4j-1.2.16.jar:/shared/hive/lib/mysql-connector-java-5.1.18-bin.jar:/shared/hive/lib/slf4j-api-1.6.1.jar:/shared/hive/lib/slf4j-log4j12-1.6.1.jar:/shared/hive/lib/stringtemplate-3.1-b1.jar:/shared/hive/lib/zookeeper-3.4.3.jar:
But even so:

   - the log dir is still wrong (writing to /tmp/${user}/hive.log instead
   of $HIVE_HOME/logs)
   - the following message in the log file

2012-11-30 00:12:31,775 WARN  conf.HiveConf
(HiveConf.java:<clinit>(70)) - *hive-site.xml
not found on CLASSPATH*
* *
*
*
2012/11/30 Bing Li <[EMAIL PROTECTED]>

> which version of hive do you use?
>
> Could you try to add the following debug line in bin/hive before hive real
> executes, and see the result?
>
> *echo "CLASSPATH=$CLASSPATH"*
>
> if [ "$TORUN" = "" ]; then
>    echo "Service $SERVICE not found"
>    echo "Available Services: $SERVICE_LIST"
>    exit 7
> else
>    $TORUN "$@"
> fi
>
> The version I used is 0.9.0
>
>
>
> 2012/11/30 Stephen Boesch <[EMAIL PROTECTED]>
>
>> Yes i do mean the log is in the wrong location, since it was set to a
>> persistent path in the $HIVE_CONF_DIR/lhive-log4j.properties.
>>
>> None of the files in that directory appear to be picked up properly:
>> neither the hive-site.xml nor log4j.properties.
>>
>> I have put echo statements into the 'hive" and the hive-config.sh  shell

You can see if the classpath is being passed correctly to hadoop by putting in an echo statement around line 150 of the hive cli script where it passes the CLASSPATH variable to HADOOP_CLASSPATH.
# pass classpath to hadoop
export HADOOP_CLASSPATH="${HADOOP_CLASSPATH}:${CLASSPATH}"

You could also echo the classpath in the hadoop script (in your HADOOP_HOME\bin directory) to see if the classpath is being passed correctly to the time when the cli jar is invoked.

As far as the logs location, if this is not set in your hive-site.xml, you can set it by passing  in HIVE_OPTS when you invoke the command line.

Like so:
EXPORT HIVE_OPTS= -hiveconf hive.log.dir=$ HIVE_HOME\logs"
Then run "hive"

Or:
Run "hive --hiveconf hive.log.dir=$ HIVE_HOME\logs"

Thanks,
Lauren
From: Stephen Boesch [mailto:[EMAIL PROTECTED]]
Sent: Friday, November 30, 2012 12:16 AM
To: [EMAIL PROTECTED]
Subject: Re: hive-site.xml not found on classpath

running 0.9.0 (you can see it from the classpath shown below);

steve@mithril:/shared/cdh4$ echo $HIVE_CONF_DIR
/shared/hive/conf
steve@mithril:/shared/cdh4$ ls -l $HIVE_CONF_DIR
total 152
-rw-r--r-- 1 steve steve 46053 2011-12-13 00:36 hive-default.xml.template
-rw-r--r-- 1 steve steve  1615 2012-11-13 23:37 hive-env.bullshit.sh<http://hive-env.bullshit.sh>
-rw-r--r-- 1 steve steve  1671 2012-11-28 01:43 hive-env.sh
-rw-r--r-- 1 steve steve  1593 2011-12-13 00:36 hive-env.sh.template
-rw-r--r-- 1 steve steve  1637 2011-12-13 00:36 hive-exec-log4j.properties.template
-rw-r--r-- 1 root  root   2056 2012-11-28 01:38 hive-log4j.properties
-rw-r--r-- 1 steve steve  2056 2012-03-25 12:49 hive-log4j.properties.template
-rw-r--r-- 1 steve steve  4415 2012-11-25 23:02 hive-site.xml
steve@mithril:/shared/cdh4$ echo $HIVE_HOME
/shared/hive
steve@mithril:/shared/cdh4$ echo $(which hive)
/shared/hive/bin/hive

also you can see the hive/conf is the first entry

After adding the debug statement:

classpath=/shared/hive/conf:/shared/hive/lib/antlr-runtime-3.0.1.jar:/shared/hive/lib/commons-cli-1.2.jar:/shared/hive/lib/commons-codec-1.3.jar:/shared/hive/lib/commons-collections-3.2.1.jar:/shared/hive/lib/commons-dbcp-1.4.jar:/shared/hive/lib/commons-lang-2.4.jar:/shared/hive/lib/commons-logging-1.0.4.jar:/shared/hive/lib/commons-logging-api-1.0.4.jar:/shared/hive/lib/commons-pool-1.5.4.jar:/shared/hive/lib/datanucleus-connectionpool-2.0.3.jar:/shared/hive/lib/datanucleus-core-2.0.3.jar:/shared/hive/lib/datanucleus-enhancer-2.0.3.jar:/shared/hive/lib/datanucleus-rdbms-2.0.3.jar:/shared/hive/lib/derby-10.4.2.0.jar:/shared/hive/lib/guava-r09.jar:/shared/hive/lib/hbase-0.92.0.jar:/shared/hive/lib/hbase-0.92.0-tests.jar:/shared/hive/lib/hive-builtins-0.9.0.jar:/shared/hive/lib/hive-cli-0.9.0.jar:/shared/hive/lib/hive-common-0.9.0.jar:/shared/hive/lib/hive-contrib-0.9.0.jar:/shared/hive/lib/hive_contrib.jar:/shared/hive/lib/hive-exec-0.9.0.jar:/shared/hive/lib/hive-hbase-handler-0.9.0.jar:/shared/hive/lib/hive-hwi-0.9.0.jar:/shared/hive/lib/hive-jdbc-0.9.0.jar:/shared/hive/lib/hive-metastore-0.9.0.jar:/shared/hive/lib/hive-pdk-0.9.0.jar:/shared/hive/lib/hive-serde-0.9.0.jar:/shared/hive/lib/hive-service-0.9.0.jar:/shared/hive/lib/hive-shims-0.9.0.jar:/shared/hive/lib/jackson-core-asl-1.8.8.jar:/shared/hive/lib/jackson-jaxrs-1.8.8.jar:/shared/hive/lib/jackson-mapper-asl-1.8.8.jar:/shared/hive/lib/jackson-xc-1.8.8.jar:/shared/hive/lib/JavaEWAH-0.3.2.jar:/shared/hive/lib/jdo2-api-2.3-ec.jar:/shared/hive/lib/jline-0.9.94.jar:/shared/hive/lib/json-20090211.jar:/shared/hive/lib/libfb303-0.7.0.jar:/shared/hive/lib/libfb303.jar:/shared/hive/lib/libthrift-0.7.0.jar:/shared/hive/lib/libthrift.jar:/shared/hive/lib/log4j-1.2.16.jar:/shared/hive/lib/mysql-connector-java-5.1.18-bin.jar:/shared/hive/lib/slf4j-api-1.6.1.jar:/shared/hive/lib/slf4j-log4j12-1.6.1.jar:/shared/hive/lib/stringtemplate-3.1-b1.jar:/shared/hive/lib/zookeeper-3.4.3.jar:
But even so:

  *   the log dir is still wrong (writing to /tmp/${user}/hive.log instead of $HIVE_HOME/logs)
  *   the following message in the log file
2012-11-30 00:12:31,775 WARN  conf.HiveConf (HiveConf.java:<clinit>(70)) - hive-site.xml not found on CLASSPATH

2012/11/30 Bing Li <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
which version of hive do you use?

Could you try to add the following debug line in bin/hive before hive real executes, and see the result?

echo "CLASSPATH=$CLASSPATH"

if [ "$TORUN" = "" ]; then
   echo "Service $SERVICE not found"
   echo "Available Services: $SERVICE_LIST"
   exit 7
else
   $TORUN "$@"
fi

The version I used is 0.9.0
2012/11/30 Stephen Boesch <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Yes i do mean the log is in the wrong location, since it was set to a persistent path in the $HIVE_CONF_DIR/lhive-log4j.properties.

None of the files in that directory appear to be picked up properly: neither the hive-site.xml nor log4j.properties.

I have put echo statements into the 'hive" and the hive-config.sh  shell scripts and the echo statements prove that  HIVE_CONF_DIR is set properly:  /shared/hive/conf

But even so the following problems occur:

  *   the message "hive-site.xml is not found in the classpath"
  *   none of the hive-site.xml values are taking properly
  *   the log4j.properties in that same directory is not taking effect.
2012/11/29 Bing Li <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Hi, Stephen
what did you mean the "wrong place under /tmp" in
"I am seeing the following message in the logs (which are in the wrong place under /tmp..)" ?

Did you mean that you set a different log dir but it didn't work?

the log dir should be set in conf/hive-log4j.properties, conf/hive-exec-log4j.properties
and you can try to reset HIVE_CONF_DIR in conf/hive-env.sh with 'export" command.

- Bing

2012/11/30 Stephen Boesch <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
thought i mentioned in the posts those were already set and verified.. but yes in any case that's first thing looked at.

steve@mithri

The first element of the classpath is the right one already.. but I STILL
get the hive-site.xml is not found in classpath.  Only hive gives me
issues.  hdfs, mapred, hbase are all running fine.

HADOOP_CLASSPATH=:*/shared/hive/conf:*
/shared/hive/lib/antlr-runtime-3.0.1.jar:/shared/hive/lib/commons-cli-1.2.jar:/shared/hive/lib/commons-codec-1.3.jar:/shared/hive/lib/commons-collections-3.2.1.jar:/shared/hive/lib/commons-dbcp-1.4.jar:/shared/hive/lib/commons-lang-2.4.jar:/shared/hive/lib/commons-logging-1.0.4.jar:/shared/hive/lib/commons-logging-api-1.0.4.jar:/shared/hive/lib/commons-pool-1.5.4.jar:/shared/hive/lib/datanucleus-connectionpool-2.0.3.jar:/shared/hive/lib/datanucleus-core-2.0.3.jar:/shared/hive/lib/datanucleus-enhancer-2.0.3.jar:/shared/hive/lib/datanucleus-rdbms-2.0.3.jar:/shared/hive/lib/derby-10.4.2.0.jar:/shared/hive/lib/guava-r09.jar:/shared/hive/lib/hbase-0.92.0.jar:/shared/hive/lib/hbase-0.92.0-tests.jar:/shared/hive/lib/hive-builtins-0.9.0.jar:/shared/hive/lib/hive-cli-0.9.0.jar:/shared/hive/lib/hive-common-0.9.0.jar:/shared/hive/lib/hive-contrib-0.9.0.jar:/shared/hive/lib/hive_contrib.jar:/shared/hive/lib/hive-exec-0.9.0.jar:/shared/hive/lib/hive-hbase-handler-0.9.0.jar:/shared/hive/lib/hive-hwi-0.9.0.jar:/shared/hive/lib/hive-jdbc-0.9.0.jar:/shared/hive/lib/hive-metastore-0.9.0.jar:/shared/hive/lib/hive-pdk-0.9.0.jar:/shared/hive/lib/hive-serde-0.9.0.jar:/shared/hive/lib/hive-service-0.9.0.jar:/shared/hive/lib/hive-shims-0.9.0.jar:/shared/hive/lib/jackson-core-asl-1.8.8.jar:/shared/hive/lib/jackson-jaxrs-1.8.8.jar:/shared/hive/lib/jackson-mapper-asl-1.8.8.jar:/shared/hive/lib/jackson-xc-1.8.8.jar:/shared/hive/lib/JavaEWAH-0.3.2.jar:/shared/hive/lib/jdo2-api-2.3-ec.jar:/shared/hive/lib/jline-0.9.94.

steve@mithril:/shared/hive/bin$ *ls -lrta /shared/hive/conf/hive-site.xml*
-rw-r--r-- 1 steve steve 4415 2012-11-25 23:02
/shared/hive/conf/hive-site.xml

2012/11/30 Lauren Yang <[EMAIL PROTECTED]>

>  You can see if the classpath is being passed correctly to hadoop by
> putting in an echo statement around line 150 of the hive cli script where
> it passes the CLASSPATH variable to HADOOP_CLASSPATH.****
>
> # pass classpath to hadoop****
>
> export HADOOP_CLASSPATH="${HADOOP_CLASSPATH}:${CLASSPATH}"****
>
> ** **
>
> You could also echo the classpath in the hadoop script (in your
> HADOOP_HOME\bin directory) to see if the classpath is being passed
> correctly to the time when the cli jar is invoked.****
>
> ** **
>
> As far as the logs location, if this is not set in your hive-site.xml, you
> can set it by passing  in HIVE_OPTS when you invoke the command line.****
>
> ** **
>
> Like so:****
>
> EXPORT HIVE_OPTS= -hiveconf hive.log.dir=$ HIVE_HOME\logs"****
>
> Then run “hive”****
>
> ** **
>
> Or:****
>
> Run “hive --hiveconf hive.log.dir=$ HIVE_HOME\logs”****
>
> ** **
>
> Thanks,****
>
> Lauren****
>
> ** **
>
> ** **
>
> *From:* Stephen Boesch [mailto:[EMAIL PROTECTED]]
> *Sent:* Friday, November 30, 2012 12:16 AM
> *To:* [EMAIL PROTECTED]
> *Subject:* Re: hive-site.xml not found on classpath****
>
> ** **
>
> running 0.9.0 (you can see it from the classpath shown below);****
>
> ** **
>
> steve@mithril:/shared/cdh4$ echo $HIVE_CONF_DIR****
>
> /shared/hive/conf****
>
> steve@mithril:/shared/cdh4$ ls -l $HIVE_CONF_DIR****
>
> total 152****
>
> -rw-r--r-- 1 steve steve 46053 2011-12-13 00:36 hive-default.xml.template*
> ***
>
> -rw-r--r-- 1 steve steve  1615 2012-11-13 23:37 hive-env.bullshit.sh****
>
> -rw-r--r-- 1 steve steve  1671 2012-11-28 01:43 hive-env.sh****
>
> -rw-r--r-- 1 steve steve  1593 2011-12-13 00:36 hive-env.sh.template****
>
> -rw-r--r-- 1 steve steve  1637 2011-12-13 00:36
> hive-exec-log4j.properties.template****
>
> -rw-r--r-- 1 root  root   2056 2012-11-28 01:38 hive-log4j.properties****
>
> -rw-r--r-- 1 steve steve  2056 2012-03-25 12:49
> hive-log4j.properties.template****
>
> -rw-r--r-- 1 steve steve  4415 2012-11-25 23:02 hive-site.xml****
>
> steve@mithril:/shared/cdh4$ echo $HIVE_HOME****

I ended up patching the HiveConf.java  . If  hive-site.xml were not found
on the classpath then:

   - an   o.a.h.fs.Path object is created from
    System.getenv("HIVE_CONF_DIR") + File.seperator + "hive-site.xml"
   - the Path is sent to the base class Configuration.addResource - whiich
   btw accepts either resourceURL's or Path objects

the problem was resolved  with that change to HiveConf.java:  the
hive-site.xml is loaded properly via the Path (instead of URL)  and as a
consequence the  jdoConnectionURL and related properties for the mysql
metastore are loaded allowing the metadata to be stored in mysql

2012/12/9 Stephen Boesch <[EMAIL PROTECTED]>

> The first element of the classpath is the right one already.. but I STILL
> get the hive-site.xml is not found in classpath.  Only hive gives me
> issues.  hdfs, mapred, hbase are all running fine.
>
> HADOOP_CLASSPATH=:*/shared/hive/conf:*
> /shared/hive/lib/antlr-runtime-3.0.1.jar:/shared/hive/lib/commons-cli-1.2.jar:/shared/hive/lib/commons-codec-1.3.jar:/shared/hive/lib/commons-collections-3.2.1.jar:/shared/hive/lib/commons-dbcp-1.4.jar:/shared/hive/lib/commons-lang-2.4.jar:/shared/hive/lib/commons-logging-1.0.4.jar:/shared/hive/lib/commons-logging-api-1.0.4.jar:/shared/hive/lib/commons-pool-1.5.4.jar:/shared/hive/lib/datanucleus-connectionpool-2.0.3.jar:/shared/hive/lib/datanucleus-core-2.0.3.jar:/shared/hive/lib/datanucleus-enhancer-2.0.3.jar:/shared/hive/lib/datanucleus-rdbms-2.0.3.jar:/shared/hive/lib/derby-10.4.2.0.jar:/shared/hive/lib/guava-r09.jar:/shared/hive/lib/hbase-0.92.0.jar:/shared/hive/lib/hbase-0.92.0-tests.jar:/shared/hive/lib/hive-builtins-0.9.0.jar:/shared/hive/lib/hive-cli-0.9.0.jar:/shared/hive/lib/hive-common-0.9.0.jar:/shared/hive/lib/hive-contrib-0.9.0.jar:/shared/hive/lib/hive_contrib.jar:/shared/hive/lib/hive-exec-0.9.0.jar:/shared/hive/lib/hive-hbase-handler-0.9.0.jar:/shared/hive/lib/hive-hwi-0.9.0.jar:/shared/hive/lib/hive-jdbc-0.9.0.jar:/shared/hive/lib/hive-metastore-0.9.0.jar:/shared/hive/lib/hive-pdk-0.9.0.jar:/shared/hive/lib/hive-serde-0.9.0.jar:/shared/hive/lib/hive-service-0.9.0.jar:/shared/hive/lib/hive-shims-0.9.0.jar:/shared/hive/lib/jackson-core-asl-1.8.8.jar:/shared/hive/lib/jackson-jaxrs-1.8.8.jar:/shared/hive/lib/jackson-mapper-asl-1.8.8.jar:/shared/hive/lib/jackson-xc-1.8.8.jar:/shared/hive/lib/JavaEWAH-0.3.2.jar:/shared/hive/lib/jdo2-api-2.3-ec.jar:/shared/hive/lib/jline-0.9.94.
>
> steve@mithril:/shared/hive/bin$ *ls -lrta /shared/hive/conf/hive-site.xml*
> -rw-r--r-- 1 steve steve 4415 2012-11-25 23:02
> /shared/hive/conf/hive-site.xml
>
>
>
> 2012/11/30 Lauren Yang <[EMAIL PROTECTED]>
>
>>  You can see if the classpath is being passed correctly to hadoop by
>> putting in an echo statement around line 150 of the hive cli script where
>> it passes the CLASSPATH variable to HADOOP_CLASSPATH.****
>>
>> # pass classpath to hadoop****
>>
>> export HADOOP_CLASSPATH="${HADOOP_CLASSPATH}:${CLASSPATH}"****
>>
>> ** **
>>
>> You could also echo the classpath in the hadoop script (in your
>> HADOOP_HOME\bin directory) to see if the classpath is being passed
>> correctly to the time when the cli jar is invoked.****
>>
>> ** **
>>
>> As far as the logs location, if this is not set in your hive-site.xml,
>> you can set it by passing  in HIVE_OPTS when you invoke the command line.
>> ****
>>
>> ** **
>>
>> Like so:****
>>
>> EXPORT HIVE_OPTS= -hiveconf hive.log.dir=$ HIVE_HOME\logs"****
>>
>> Then run “hive”****
>>
>> ** **
>>
>> Or:****
>>
>> Run “hive --hiveconf hive.log.dir=$ HIVE_HOME\logs”****
>>
>> ** **
>>
>> Thanks,****
>>
>> Lauren****
>>
>> ** **
>>
>> ** **
>>
>> *From:* Stephen Boesch [mailto:[EMAIL PROTECTED]]
>> *Sent:* Friday, November 30, 2012 12:16 AM
>> *To:* [EMAIL PROTECTED]
>> *Subject:* Re: hive-site.xml not found on classpath****
>>
>> ** **
>>
>> running 0.9.0 (you can see it from the classpath shown below);****
>>
>> ** **
>>
>> steve@mithril:/shared/cdh4$ echo $HIVE_CONF_DIR****
>>
>> /shared/hive/conf****