hi,

I'm try to use postgres as stats database. And made following settings
in hive-site.xml
<property>
  <name>hive.stats.dbclass</name>
  <value>jdbc:postgresql</value>
  <description>The default database that stores temporary hive
statistics.</description>
</property>

<property>
  <name>hive.stats.autogather</name>
  <value>true</value>
  <description>A flag to gather statistics automatically during the
INSERT OVERWRITE command.</description>
</property>

<property>
  <name>hive.stats.jdbcdriver</name>
  <value>org.postgresql.Driver</value>
  <description>The JDBC driver for the database that stores temporary
hive statistics.</description>
</property>

<property>
  <name>hive.stats.dbconnectionstring</name>
  <value>jdbc:postgresql://localhost/hive_statsdb?createDatabaseIfNotExist=true;user=hive;password=pwd</value>
  <description>The default connection string for the database that
stores temporary hive statistics.</description>
</property>

I use postgres as hive meta database, so there is a
postgresql-9.0-801.jdbc4.jar file in lib.

After run 'analyse table t1 partitions(dt) comput statistics;' in hive
cli, it will output some stats info in cli, but nothing in db. And I
can found there is the flowing errors

1-08-15 14:54:54,767 INFO
org.apache.hadoop.hive.ql.exec.TableScanOperator: Stats Gathering
found a new partition spec = dt=20110805
2011-08-15 14:54:54,767 INFO
org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 forwarding 1 rows
2011-08-15 14:54:54,767 INFO ExecMapper: ExecMapper: processing 1
rows: used memory = 39953640
2011-08-15 14:54:54,768 INFO
org.apache.hadoop.hive.ql.exec.MapOperator: 1 finished. closing...
2011-08-15 14:54:54,768 INFO
org.apache.hadoop.hive.ql.exec.MapOperator: 1 forwarded 2 rows
2011-08-15 14:54:54,768 INFO
org.apache.hadoop.hive.ql.exec.MapOperator: DESERIALIZE_ERRORS:0
2011-08-15 14:54:54,768 INFO
org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 finished.
closing...
2011-08-15 14:54:54,768 INFO
org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 forwarded 2 rows
2011-08-15 14:54:54,772 ERROR
org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsPublisher: Error during
JDBC connection to
jdbc:postgresql://localhost/hive_statsdb?createDatabaseIfNotExist=true;user=hive;password=pwd.
java.lang.ClassNotFoundException: org.postgresql.Driver
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:169)
at org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsPublisher.connect(JDBCStatsPublisher.java:55)
at org.apache.hadoop.hive.ql.exec.TableScanOperator.publishStats(TableScanOperator.java:202)
at org.apache.hadoop.hive.ql.exec.TableScanOperator.closeOp(TableScanOperator.java:164)
at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:557)
at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566)
at org.apache.hadoop.hive.ql.exec.ExecMapper.close(ExecMapper.java:193)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
2011-08-15 14:54:54,774 INFO
org.apache.hadoop.hive.ql.exec.TableScanOperator: StatsPublishing
error: cannot connect to database.
2011-08-15 14:54:54,774 INFO
org.apache.hadoop.hive.ql.exec.MapOperator: 1 Close done

oh, found hive only support mysql and hbase. I'll try hbase.

On Mon, Aug 15, 2011 at 3:09 PM, wd <[EMAIL PROTECTED]> wrote:
> hi,
>
> I'm try to use postgres as stats database. And made following settings
> in hive-site.xml
>
>
> <property>
>  <name>hive.stats.dbclass</name>
>  <value>jdbc:postgresql</value>
>  <description>The default database that stores temporary hive
> statistics.</description>
> </property>
>
> <property>
>  <name>hive.stats.autogather</name>
>  <value>true</value>
>  <description>A flag to gather statistics automatically during the
> INSERT OVERWRITE command.</description>
> </property>
>
> <property>
>  <name>hive.stats.jdbcdriver</name>
>  <value>org.postgresql.Driver</value>
>  <description>The JDBC driver for the database that stores temporary
> hive statistics.</description>
> </property>
>
> <property>
>  <name>hive.stats.dbconnectionstring</name>
>  <value>jdbc:postgresql://localhost/hive_statsdb?createDatabaseIfNotExist=true;user=hive;password=pwd</value>
>  <description>The default connection string for the database that
> stores temporary hive statistics.</description>
> </property>
>
> I use postgres as hive meta database, so there is a
> postgresql-9.0-801.jdbc4.jar file in lib.
>
> After run 'analyse table t1 partitions(dt) comput statistics;' in hive
> cli, it will output some stats info in cli, but nothing in db. And I
> can found there is the flowing errors
>
> 1-08-15 14:54:54,767 INFO
> org.apache.hadoop.hive.ql.exec.TableScanOperator: Stats Gathering
> found a new partition spec = dt=20110805
> 2011-08-15 14:54:54,767 INFO
> org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 forwarding 1 rows
> 2011-08-15 14:54:54,767 INFO ExecMapper: ExecMapper: processing 1
> rows: used memory = 39953640
> 2011-08-15 14:54:54,768 INFO
> org.apache.hadoop.hive.ql.exec.MapOperator: 1 finished. closing...
> 2011-08-15 14:54:54,768 INFO
> org.apache.hadoop.hive.ql.exec.MapOperator: 1 forwarded 2 rows
> 2011-08-15 14:54:54,768 INFO
> org.apache.hadoop.hive.ql.exec.MapOperator: DESERIALIZE_ERRORS:0
> 2011-08-15 14:54:54,768 INFO
> org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 finished.
> closing...
> 2011-08-15 14:54:54,768 INFO
> org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 forwarded 2 rows
> 2011-08-15 14:54:54,772 ERROR
> org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsPublisher: Error during
> JDBC connection to
> jdbc:postgresql://localhost/hive_statsdb?createDatabaseIfNotExist=true;user=hive;password=pwd.
> java.lang.ClassNotFoundException: org.postgresql.Driver
>        at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>        at java.security.AccessController.doPrivileged(Native Method)
>        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>        at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>        at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>        at java.lang.Class.forName0(Native Method)
>        at java.lang.Class.forName(Class.java:169)
>        at org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsPublisher.connect(JDBCStatsPublisher.java:55)
>        at org.apache.hadoop.hive.ql.exec.TableScanOperator.publishStats(TableScanOperator.java:202)
>        at org.apache.hadoop.hive.ql.exec.TableScanOperator.closeOp(TableScanOperator.java:164)
>        at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:557)
>        at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566)
>        at org.apache.hadoop.hive.ql.exec.ExecMapper.close(ExecMapper.java:193)
>        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
>        at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
>        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
>        at org.apache.hadoop.mapred.Child.main(Child.java:170)
> 2011-08-15 14:54:54,774 INFO
> org.apache.hadoop.hive.ql.exec.TableScanOperator: StatsPublishing
> error: cannot connect to database.

HBase Publisher/Aggregator classes cannot be loaded.

need to configure publisher/aggregator for hbase...there is only one
way, that is use mysql ..

does stats database will optimize hive query? Consider whether or not
setup a mysql for this.

On Mon, Aug 15, 2011 at 3:17 PM, wd <[EMAIL PROTECTED]> wrote:
> oh, found hive only support mysql and hbase. I'll try hbase.
>
> On Mon, Aug 15, 2011 at 3:09 PM, wd <[EMAIL PROTECTED]> wrote:
>> hi,
>>
>> I'm try to use postgres as stats database. And made following settings
>> in hive-site.xml
>>
>>
>> <property>
>>  <name>hive.stats.dbclass</name>
>>  <value>jdbc:postgresql</value>
>>  <description>The default database that stores temporary hive
>> statistics.</description>
>> </property>
>>
>> <property>
>>  <name>hive.stats.autogather</name>
>>  <value>true</value>
>>  <description>A flag to gather statistics automatically during the
>> INSERT OVERWRITE command.</description>
>> </property>
>>
>> <property>
>>  <name>hive.stats.jdbcdriver</name>
>>  <value>org.postgresql.Driver</value>
>>  <description>The JDBC driver for the database that stores temporary
>> hive statistics.</description>
>> </property>
>>
>> <property>
>>  <name>hive.stats.dbconnectionstring</name>
>>  <value>jdbc:postgresql://localhost/hive_statsdb?createDatabaseIfNotExist=true;user=hive;password=pwd</value>
>>  <description>The default connection string for the database that
>> stores temporary hive statistics.</description>
>> </property>
>>
>> I use postgres as hive meta database, so there is a
>> postgresql-9.0-801.jdbc4.jar file in lib.
>>
>> After run 'analyse table t1 partitions(dt) comput statistics;' in hive
>> cli, it will output some stats info in cli, but nothing in db. And I
>> can found there is the flowing errors
>>
>> 1-08-15 14:54:54,767 INFO
>> org.apache.hadoop.hive.ql.exec.TableScanOperator: Stats Gathering
>> found a new partition spec = dt=20110805
>> 2011-08-15 14:54:54,767 INFO
>> org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 forwarding 1 rows
>> 2011-08-15 14:54:54,767 INFO ExecMapper: ExecMapper: processing 1
>> rows: used memory = 39953640
>> 2011-08-15 14:54:54,768 INFO
>> org.apache.hadoop.hive.ql.exec.MapOperator: 1 finished. closing...
>> 2011-08-15 14:54:54,768 INFO
>> org.apache.hadoop.hive.ql.exec.MapOperator: 1 forwarded 2 rows
>> 2011-08-15 14:54:54,768 INFO
>> org.apache.hadoop.hive.ql.exec.MapOperator: DESERIALIZE_ERRORS:0
>> 2011-08-15 14:54:54,768 INFO
>> org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 finished.
>> closing...
>> 2011-08-15 14:54:54,768 INFO
>> org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 forwarded 2 rows
>> 2011-08-15 14:54:54,772 ERROR
>> org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsPublisher: Error during
>> JDBC connection to
>> jdbc:postgresql://localhost/hive_statsdb?createDatabaseIfNotExist=true;user=hive;password=pwd.
>> java.lang.ClassNotFoundException: org.postgresql.Driver
>>        at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>>        at java.security.AccessController.doPrivileged(Native Method)
>>        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>>        at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>>        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>>        at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>>        at java.lang.Class.forName0(Native Method)
>>        at java.lang.Class.forName(Class.java:169)
>>        at org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsPublisher.connect(JDBCStatsPublisher.java:55)
>>        at org.apache.hadoop.hive.ql.exec.TableScanOperator.publishStats(TableScanOperator.java:202)
>>        at org.apache.hadoop.hive.ql.exec.TableScanOperator.closeOp(TableScanOperator.java:164)
>>        at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:557)
>>        at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566)
>>        at org.apache.hadoop.hive.ql.exec.ExecMapper.close(ExecMapper.java:193)
>>        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)

Hi,wd

You should configure "hive.stats.dbconnectionstring" as follows.

<property>
 <name>hive.stats.dbconnectionstring</name>
 <value>jdbc:postgresql://localhost/hive_statsdb?createDatabaseIfNotExist=true&user=hive&password=pwd</value>
 <description>The default connection string for the database that
stores temporary hive statistics.</description>
</property>

Regards,

Jander.
On Mon, Aug 15, 2011 at 3:09 PM, wd <[EMAIL PROTECTED]> wrote:

> hi,
>
> I'm try to use postgres as stats database. And made following settings
> in hive-site.xml
>
>
> <property>
>  <name>hive.stats.dbclass</name>
>  <value>jdbc:postgresql</value>
>  <description>The default database that stores temporary hive
> statistics.</description>
> </property>
>
> <property>
>  <name>hive.stats.autogather</name>
>  <value>true</value>
>  <description>A flag to gather statistics automatically during the
> INSERT OVERWRITE command.</description>
> </property>
>
> <property>
>  <name>hive.stats.jdbcdriver</name>
>  <value>org.postgresql.Driver</value>
>  <description>The JDBC driver for the database that stores temporary
> hive statistics.</description>
> </property>
>
> <property>
>  <name>hive.stats.dbconnectionstring</name>
>
>  <value>jdbc:postgresql://localhost/hive_statsdb?createDatabaseIfNotExist=true;user=hive;password=pwd</value>
>  <description>The default connection string for the database that
> stores temporary hive statistics.</description>
> </property>
>
> I use postgres as hive meta database, so there is a
> postgresql-9.0-801.jdbc4.jar file in lib.
>
> After run 'analyse table t1 partitions(dt) comput statistics;' in hive
> cli, it will output some stats info in cli, but nothing in db. And I
> can found there is the flowing errors
>
> 1-08-15 14:54:54,767 INFO
> org.apache.hadoop.hive.ql.exec.TableScanOperator: Stats Gathering
> found a new partition spec = dt=20110805
> 2011-08-15 14:54:54,767 INFO
> org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 forwarding 1 rows
> 2011-08-15 14:54:54,767 INFO ExecMapper: ExecMapper: processing 1
> rows: used memory = 39953640
> 2011-08-15 14:54:54,768 INFO
> org.apache.hadoop.hive.ql.exec.MapOperator: 1 finished. closing...
> 2011-08-15 14:54:54,768 INFO
> org.apache.hadoop.hive.ql.exec.MapOperator: 1 forwarded 2 rows
> 2011-08-15 14:54:54,768 INFO
> org.apache.hadoop.hive.ql.exec.MapOperator: DESERIALIZE_ERRORS:0
> 2011-08-15 14:54:54,768 INFO
> org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 finished.
> closing...
> 2011-08-15 14:54:54,768 INFO
> org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 forwarded 2 rows
> 2011-08-15 14:54:54,772 ERROR
> org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsPublisher: Error during
> JDBC connection to
>
> jdbc:postgresql://localhost/hive_statsdb?createDatabaseIfNotExist=true;user=hive;password=pwd.
> java.lang.ClassNotFoundException: org.postgresql.Driver
>        at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>        at java.security.AccessController.doPrivileged(Native Method)
>        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>        at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>        at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>        at java.lang.Class.forName0(Native Method)
>        at java.lang.Class.forName(Class.java:169)
>        at
> org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsPublisher.connect(JDBCStatsPublisher.java:55)
>        at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.publishStats(TableScanOperator.java:202)
>        at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.closeOp(TableScanOperator.java:164)
>        at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:557)
>        at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566)
>        at
> org.apache.hadoop.hive.ql.exec.ExecMapper.close(ExecMapper.java:193)
>        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)

Thanks,
Jander

The error in log is 'java.lang.ClassNotFoundException:
org.postgresql.Driver', not can't connect or user name or password error.

On Wed, Aug 17, 2011 at 3:53 PM, Jander g <[EMAIL PROTECTED]> wrote:

> Hi,wd
>
> You should configure "hive.stats.dbconnectionstring" as follows.
>
>
> <property>
>  <name>hive.stats.dbconnectionstring</name>
>
>  <value>jdbc:postgresql://localhost/hive_statsdb?createDatabaseIfNotExist=true&user=hive&password=pwd</value>
>
>  <description>The default connection string for the database that
> stores temporary hive statistics.</description>
> </property>
>
> Regards,
>
> Jander.
>
>
> On Mon, Aug 15, 2011 at 3:09 PM, wd <[EMAIL PROTECTED]> wrote:
>
>> hi,
>>
>> I'm try to use postgres as stats database. And made following settings
>> in hive-site.xml
>>
>>
>> <property>
>>  <name>hive.stats.dbclass</name>
>>  <value>jdbc:postgresql</value>
>>  <description>The default database that stores temporary hive
>> statistics.</description>
>> </property>
>>
>> <property>
>>  <name>hive.stats.autogather</name>
>>  <value>true</value>
>>  <description>A flag to gather statistics automatically during the
>> INSERT OVERWRITE command.</description>
>> </property>
>>
>> <property>
>>  <name>hive.stats.jdbcdriver</name>
>>  <value>org.postgresql.Driver</value>
>>  <description>The JDBC driver for the database that stores temporary
>> hive statistics.</description>
>> </property>
>>
>> <property>
>>  <name>hive.stats.dbconnectionstring</name>
>>
>>  <value>jdbc:postgresql://localhost/hive_statsdb?createDatabaseIfNotExist=true;user=hive;password=pwd</value>
>>  <description>The default connection string for the database that
>> stores temporary hive statistics.</description>
>> </property>
>>
>> I use postgres as hive meta database, so there is a
>> postgresql-9.0-801.jdbc4.jar file in lib.
>>
>> After run 'analyse table t1 partitions(dt) comput statistics;' in hive
>> cli, it will output some stats info in cli, but nothing in db. And I
>> can found there is the flowing errors
>>
>> 1-08-15 14:54:54,767 INFO
>> org.apache.hadoop.hive.ql.exec.TableScanOperator: Stats Gathering
>> found a new partition spec = dt=20110805
>> 2011-08-15 14:54:54,767 INFO
>> org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 forwarding 1 rows
>> 2011-08-15 14:54:54,767 INFO ExecMapper: ExecMapper: processing 1
>> rows: used memory = 39953640
>> 2011-08-15 14:54:54,768 INFO
>> org.apache.hadoop.hive.ql.exec.MapOperator: 1 finished. closing...
>> 2011-08-15 14:54:54,768 INFO
>> org.apache.hadoop.hive.ql.exec.MapOperator: 1 forwarded 2 rows
>> 2011-08-15 14:54:54,768 INFO
>> org.apache.hadoop.hive.ql.exec.MapOperator: DESERIALIZE_ERRORS:0
>> 2011-08-15 14:54:54,768 INFO
>> org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 finished.
>> closing...
>> 2011-08-15 14:54:54,768 INFO
>> org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 forwarded 2 rows
>> 2011-08-15 14:54:54,772 ERROR
>> org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsPublisher: Error during
>> JDBC connection to
>>
>> jdbc:postgresql://localhost/hive_statsdb?createDatabaseIfNotExist=true;user=hive;password=pwd.
>> java.lang.ClassNotFoundException: org.postgresql.Driver
>>        at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>>        at java.security.AccessController.doPrivileged(Native Method)
>>        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>>        at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>>        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>>        at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>>        at java.lang.Class.forName0(Native Method)
>>        at java.lang.Class.forName(Class.java:169)
>>        at
>> org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsPublisher.connect(JDBCStatsPublisher.java:55)
>>        at
>> org.apache.hadoop.hive.ql.exec.TableScanOperator.publishStats(TableScanOperator.java:202)
>>        at
>> org.apache.hadoop.hive.ql.exec.TableScanOperator.closeOp(TableScanOperator.java:164)

Hi,

Iam also getting the same error. However I am using mysql for stats.

The thing is I configured mysql for metastore and it works fine and all the
metadata gets populated normally. When the metastore classes can find the
mysql jar in the class path , why cant the stats publisher find it. I looked
at the stats source and everything looks fine.

My conn string is :
jdbc:mysql://<ip>:3306/TempStatsStore&user=<name>&password=<pwd>.

Am I missing something?

Thanks

On Thu, Aug 18, 2011 at 8:19 AM, wd <[EMAIL PROTECTED]> wrote:

> The error in log is 'java.lang.ClassNotFoundException:
> org.postgresql.Driver', not can't connect or user name or password error.
>
>
> On Wed, Aug 17, 2011 at 3:53 PM, Jander g <[EMAIL PROTECTED]> wrote:
>
>> Hi,wd
>>
>> You should configure "hive.stats.dbconnectionstring" as follows.
>>
>>
>> <property>
>>  <name>hive.stats.dbconnectionstring</name>
>>
>>  <value>jdbc:postgresql://localhost/hive_statsdb?createDatabaseIfNotExist=true&user=hive&password=pwd</value>
>>
>>  <description>The default connection string for the database that
>> stores temporary hive statistics.</description>
>> </property>
>>
>> Regards,
>>
>> Jander.
>>
>>
>> On Mon, Aug 15, 2011 at 3:09 PM, wd <[EMAIL PROTECTED]> wrote:
>>
>>> hi,
>>>
>>> I'm try to use postgres as stats database. And made following settings
>>> in hive-site.xml
>>>
>>>
>>> <property>
>>>  <name>hive.stats.dbclass</name>
>>>  <value>jdbc:postgresql</value>
>>>  <description>The default database that stores temporary hive
>>> statistics.</description>
>>> </property>
>>>
>>> <property>
>>>  <name>hive.stats.autogather</name>
>>>  <value>true</value>
>>>  <description>A flag to gather statistics automatically during the
>>> INSERT OVERWRITE command.</description>
>>> </property>
>>>
>>> <property>
>>>  <name>hive.stats.jdbcdriver</name>
>>>  <value>org.postgresql.Driver</value>
>>>  <description>The JDBC driver for the database that stores temporary
>>> hive statistics.</description>
>>> </property>
>>>
>>> <property>
>>>  <name>hive.stats.dbconnectionstring</name>
>>>
>>>  <value>jdbc:postgresql://localhost/hive_statsdb?createDatabaseIfNotExist=true;user=hive;password=pwd</value>
>>>  <description>The default connection string for the database that
>>> stores temporary hive statistics.</description>
>>> </property>
>>>
>>> I use postgres as hive meta database, so there is a
>>> postgresql-9.0-801.jdbc4.jar file in lib.
>>>
>>> After run 'analyse table t1 partitions(dt) comput statistics;' in hive
>>> cli, it will output some stats info in cli, but nothing in db. And I
>>> can found there is the flowing errors
>>>
>>> 1-08-15 14:54:54,767 INFO
>>> org.apache.hadoop.hive.ql.exec.TableScanOperator: Stats Gathering
>>> found a new partition spec = dt=20110805
>>> 2011-08-15 14:54:54,767 INFO
>>> org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 forwarding 1 rows
>>> 2011-08-15 14:54:54,767 INFO ExecMapper: ExecMapper: processing 1
>>> rows: used memory = 39953640
>>> 2011-08-15 14:54:54,768 INFO
>>> org.apache.hadoop.hive.ql.exec.MapOperator: 1 finished. closing...
>>> 2011-08-15 14:54:54,768 INFO
>>> org.apache.hadoop.hive.ql.exec.MapOperator: 1 forwarded 2 rows
>>> 2011-08-15 14:54:54,768 INFO
>>> org.apache.hadoop.hive.ql.exec.MapOperator: DESERIALIZE_ERRORS:0
>>> 2011-08-15 14:54:54,768 INFO
>>> org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 finished.
>>> closing...
>>> 2011-08-15 14:54:54,768 INFO
>>> org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 forwarded 2 rows
>>> 2011-08-15 14:54:54,772 ERROR
>>> org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsPublisher: Error during
>>> JDBC connection to
>>>
>>> jdbc:postgresql://localhost/hive_statsdb?createDatabaseIfNotExist=true;user=hive;password=pwd.
>>> java.lang.ClassNotFoundException: org.postgresql.Driver
>>>        at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>>>        at java.security.AccessController.doPrivileged(Native Method)
>>>        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
Regards,
Bharath .V
w:http://researchweb.iiit.ac.in/~bharath.v

Maybe you should use
'hive.stats.jdbcdriver="org.apache.mysql.jdbc.EmbeddedDriver"'
settings?

via
http://mail-archives.apache.org/mod_mbox/hive-user/201103.mbox/%[EMAIL PROTECTED]%3E

On Fri, Aug 19, 2011 at 5:45 AM, bharath vissapragada <
[EMAIL PROTECTED]> wrote:

> Hi,
>
> Iam also getting the same error. However I am using mysql for stats.
>
> The thing is I configured mysql for metastore and it works fine and all the
> metadata gets populated normally. When the metastore classes can find the
> mysql jar in the class path , why cant the stats publisher find it. I looked
> at the stats source and everything looks fine.
>
> My conn string is :
> jdbc:mysql://<ip>:3306/TempStatsStore&user=<name>&password=<pwd>.
>
> Am I missing something?
>
> Thanks
>
>
>
>
> On Thu, Aug 18, 2011 at 8:19 AM, wd <[EMAIL PROTECTED]> wrote:
>
>> The error in log is 'java.lang.ClassNotFoundException:
>> org.postgresql.Driver', not can't connect or user name or password error.
>>
>>
>> On Wed, Aug 17, 2011 at 3:53 PM, Jander g <[EMAIL PROTECTED]> wrote:
>>
>>> Hi,wd
>>>
>>> You should configure "hive.stats.dbconnectionstring" as follows.
>>>
>>>
>>> <property>
>>>  <name>hive.stats.dbconnectionstring</name>
>>>
>>>  <value>jdbc:postgresql://localhost/hive_statsdb?createDatabaseIfNotExist=true&user=hive&password=pwd</value>
>>>
>>>  <description>The default connection string for the database that
>>> stores temporary hive statistics.</description>
>>> </property>
>>>
>>> Regards,
>>>
>>> Jander.
>>>
>>>
>>> On Mon, Aug 15, 2011 at 3:09 PM, wd <[EMAIL PROTECTED]> wrote:
>>>
>>>> hi,
>>>>
>>>> I'm try to use postgres as stats database. And made following settings
>>>> in hive-site.xml
>>>>
>>>>
>>>> <property>
>>>>  <name>hive.stats.dbclass</name>
>>>>  <value>jdbc:postgresql</value>
>>>>  <description>The default database that stores temporary hive
>>>> statistics.</description>
>>>> </property>
>>>>
>>>> <property>
>>>>  <name>hive.stats.autogather</name>
>>>>  <value>true</value>
>>>>  <description>A flag to gather statistics automatically during the
>>>> INSERT OVERWRITE command.</description>
>>>> </property>
>>>>
>>>> <property>
>>>>  <name>hive.stats.jdbcdriver</name>
>>>>  <value>org.postgresql.Driver</value>
>>>>  <description>The JDBC driver for the database that stores temporary
>>>> hive statistics.</description>
>>>> </property>
>>>>
>>>> <property>
>>>>  <name>hive.stats.dbconnectionstring</name>
>>>>
>>>>  <value>jdbc:postgresql://localhost/hive_statsdb?createDatabaseIfNotExist=true;user=hive;password=pwd</value>
>>>>  <description>The default connection string for the database that
>>>> stores temporary hive statistics.</description>
>>>> </property>
>>>>
>>>> I use postgres as hive meta database, so there is a
>>>> postgresql-9.0-801.jdbc4.jar file in lib.
>>>>
>>>> After run 'analyse table t1 partitions(dt) comput statistics;' in hive
>>>> cli, it will output some stats info in cli, but nothing in db. And I
>>>> can found there is the flowing errors
>>>>
>>>> 1-08-15 14:54:54,767 INFO
>>>> org.apache.hadoop.hive.ql.exec.TableScanOperator: Stats Gathering
>>>> found a new partition spec = dt=20110805
>>>> 2011-08-15 14:54:54,767 INFO
>>>> org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 forwarding 1 rows
>>>> 2011-08-15 14:54:54,767 INFO ExecMapper: ExecMapper: processing 1
>>>> rows: used memory = 39953640
>>>> 2011-08-15 14:54:54,768 INFO
>>>> org.apache.hadoop.hive.ql.exec.MapOperator: 1 finished. closing...
>>>> 2011-08-15 14:54:54,768 INFO
>>>> org.apache.hadoop.hive.ql.exec.MapOperator: 1 forwarded 2 rows
>>>> 2011-08-15 14:54:54,768 INFO
>>>> org.apache.hadoop.hive.ql.exec.MapOperator: DESERIALIZE_ERRORS:0
>>>> 2011-08-15 14:54:54,768 INFO
>>>> org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 finished.
>>>> closing...
>>>> 2011-08-15 14:54:54,768 INFO
>>>> org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 forwarded 2 rows
>>>> 2011-08-15 14:54:54,772 ERROR
>>>> org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsPublisher: Error during

Hi
you'd better
hive> add  derby.jar ;
before exec your sql
2011/8/18 wd <[EMAIL PROTECTED]>

> The error in log is 'java.lang.ClassNotFoundException:
> org.postgresql.Driver', not can't connect or user name or password error.
>
>
> On Wed, Aug 17, 2011 at 3:53 PM, Jander g <[EMAIL PROTECTED]> wrote:
>
>> Hi,wd
>>
>> You should configure "hive.stats.dbconnectionstring" as follows.
>>
>>
>> <property>
>>  <name>hive.stats.dbconnectionstring</name>
>>
>>  <value>jdbc:postgresql://localhost/hive_statsdb?createDatabaseIfNotExist=true&user=hive&password=pwd</value>
>>
>>  <description>The default connection string for the database that
>> stores temporary hive statistics.</description>
>> </property>
>>
>> Regards,
>>
>> Jander.
>>
>>
>> On Mon, Aug 15, 2011 at 3:09 PM, wd <[EMAIL PROTECTED]> wrote:
>>
>>> hi,
>>>
>>> I'm try to use postgres as stats database. And made following settings
>>> in hive-site.xml
>>>
>>>
>>> <property>
>>>  <name>hive.stats.dbclass</name>
>>>  <value>jdbc:postgresql</value>
>>>  <description>The default database that stores temporary hive
>>> statistics.</description>
>>> </property>
>>>
>>> <property>
>>>  <name>hive.stats.autogather</name>
>>>  <value>true</value>
>>>  <description>A flag to gather statistics automatically during the
>>> INSERT OVERWRITE command.</description>
>>> </property>
>>>
>>> <property>
>>>  <name>hive.stats.jdbcdriver</name>
>>>  <value>org.postgresql.Driver</value>
>>>  <description>The JDBC driver for the database that stores temporary
>>> hive statistics.</description>
>>> </property>
>>>
>>> <property>
>>>  <name>hive.stats.dbconnectionstring</name>
>>>
>>>  <value>jdbc:postgresql://localhost/hive_statsdb?createDatabaseIfNotExist=true;user=hive;password=pwd</value>
>>>  <description>The default connection string for the database that
>>> stores temporary hive statistics.</description>
>>> </property>
>>>
>>> I use postgres as hive meta database, so there is a
>>> postgresql-9.0-801.jdbc4.jar file in lib.
>>>
>>> After run 'analyse table t1 partitions(dt) comput statistics;' in hive
>>> cli, it will output some stats info in cli, but nothing in db. And I
>>> can found there is the flowing errors
>>>
>>> 1-08-15 14:54:54,767 INFO
>>> org.apache.hadoop.hive.ql.exec.TableScanOperator: Stats Gathering
>>> found a new partition spec = dt=20110805
>>> 2011-08-15 14:54:54,767 INFO
>>> org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 forwarding 1 rows
>>> 2011-08-15 14:54:54,767 INFO ExecMapper: ExecMapper: processing 1
>>> rows: used memory = 39953640
>>> 2011-08-15 14:54:54,768 INFO
>>> org.apache.hadoop.hive.ql.exec.MapOperator: 1 finished. closing...
>>> 2011-08-15 14:54:54,768 INFO
>>> org.apache.hadoop.hive.ql.exec.MapOperator: 1 forwarded 2 rows
>>> 2011-08-15 14:54:54,768 INFO
>>> org.apache.hadoop.hive.ql.exec.MapOperator: DESERIALIZE_ERRORS:0
>>> 2011-08-15 14:54:54,768 INFO
>>> org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 finished.
>>> closing...
>>> 2011-08-15 14:54:54,768 INFO
>>> org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 forwarded 2 rows
>>> 2011-08-15 14:54:54,772 ERROR
>>> org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsPublisher: Error during
>>> JDBC connection to
>>>
>>> jdbc:postgresql://localhost/hive_statsdb?createDatabaseIfNotExist=true;user=hive;password=pwd.
>>> java.lang.ClassNotFoundException: org.postgresql.Driver
>>>        at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>>>        at java.security.AccessController.doPrivileged(Native Method)
>>>        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>>>        at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>>>        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>>>        at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>>>        at java.lang.Class.forName0(Native Method)
>>>        at java.lang.Class.forName(Class.java:169)
>>>        at
>>> org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsPublisher.connect(JDBCStatsPublisher.java:55)
Best wishs！
Fangkun.Cao