Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Setting up stats database


Copy link to this message
-
Re: Setting up stats database
Maybe you should use
'hive.stats.jdbcdriver="org.apache.mysql.jdbc.EmbeddedDriver"'
settings?

via
http://mail-archives.apache.org/mod_mbox/hive-user/201103.mbox/%[EMAIL PROTECTED]%3E

On Fri, Aug 19, 2011 at 5:45 AM, bharath vissapragada <
[EMAIL PROTECTED]> wrote:

> Hi,
>
> Iam also getting the same error. However I am using mysql for stats.
>
> The thing is I configured mysql for metastore and it works fine and all the
> metadata gets populated normally. When the metastore classes can find the
> mysql jar in the class path , why cant the stats publisher find it. I looked
> at the stats source and everything looks fine.
>
> My conn string is :
> jdbc:mysql://<ip>:3306/TempStatsStore&user=<name>&password=<pwd>.
>
> Am I missing something?
>
> Thanks
>
>
>
>
> On Thu, Aug 18, 2011 at 8:19 AM, wd <[EMAIL PROTECTED]> wrote:
>
>> The error in log is 'java.lang.ClassNotFoundException:
>> org.postgresql.Driver', not can't connect or user name or password error.
>>
>>
>> On Wed, Aug 17, 2011 at 3:53 PM, Jander g <[EMAIL PROTECTED]> wrote:
>>
>>> Hi,wd
>>>
>>> You should configure "hive.stats.dbconnectionstring" as follows.
>>>
>>>
>>> <property>
>>>  <name>hive.stats.dbconnectionstring</name>
>>>
>>>  <value>jdbc:postgresql://localhost/hive_statsdb?createDatabaseIfNotExist=true&user=hive&password=pwd</value>
>>>
>>>  <description>The default connection string for the database that
>>> stores temporary hive statistics.</description>
>>> </property>
>>>
>>> Regards,
>>>
>>> Jander.
>>>
>>>
>>> On Mon, Aug 15, 2011 at 3:09 PM, wd <[EMAIL PROTECTED]> wrote:
>>>
>>>> hi,
>>>>
>>>> I'm try to use postgres as stats database. And made following settings
>>>> in hive-site.xml
>>>>
>>>>
>>>> <property>
>>>>  <name>hive.stats.dbclass</name>
>>>>  <value>jdbc:postgresql</value>
>>>>  <description>The default database that stores temporary hive
>>>> statistics.</description>
>>>> </property>
>>>>
>>>> <property>
>>>>  <name>hive.stats.autogather</name>
>>>>  <value>true</value>
>>>>  <description>A flag to gather statistics automatically during the
>>>> INSERT OVERWRITE command.</description>
>>>> </property>
>>>>
>>>> <property>
>>>>  <name>hive.stats.jdbcdriver</name>
>>>>  <value>org.postgresql.Driver</value>
>>>>  <description>The JDBC driver for the database that stores temporary
>>>> hive statistics.</description>
>>>> </property>
>>>>
>>>> <property>
>>>>  <name>hive.stats.dbconnectionstring</name>
>>>>
>>>>  <value>jdbc:postgresql://localhost/hive_statsdb?createDatabaseIfNotExist=true;user=hive;password=pwd</value>
>>>>  <description>The default connection string for the database that
>>>> stores temporary hive statistics.</description>
>>>> </property>
>>>>
>>>> I use postgres as hive meta database, so there is a
>>>> postgresql-9.0-801.jdbc4.jar file in lib.
>>>>
>>>> After run 'analyse table t1 partitions(dt) comput statistics;' in hive
>>>> cli, it will output some stats info in cli, but nothing in db. And I
>>>> can found there is the flowing errors
>>>>
>>>> 1-08-15 14:54:54,767 INFO
>>>> org.apache.hadoop.hive.ql.exec.TableScanOperator: Stats Gathering
>>>> found a new partition spec = dt=20110805
>>>> 2011-08-15 14:54:54,767 INFO
>>>> org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 forwarding 1 rows
>>>> 2011-08-15 14:54:54,767 INFO ExecMapper: ExecMapper: processing 1
>>>> rows: used memory = 39953640
>>>> 2011-08-15 14:54:54,768 INFO
>>>> org.apache.hadoop.hive.ql.exec.MapOperator: 1 finished. closing...
>>>> 2011-08-15 14:54:54,768 INFO
>>>> org.apache.hadoop.hive.ql.exec.MapOperator: 1 forwarded 2 rows
>>>> 2011-08-15 14:54:54,768 INFO
>>>> org.apache.hadoop.hive.ql.exec.MapOperator: DESERIALIZE_ERRORS:0
>>>> 2011-08-15 14:54:54,768 INFO
>>>> org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 finished.
>>>> closing...
>>>> 2011-08-15 14:54:54,768 INFO
>>>> org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 forwarded 2 rows
>>>> 2011-08-15 14:54:54,772 ERROR
>>>> org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsPublisher: Error during