Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Setting up stats database


Copy link to this message
-
Re: Setting up stats database
Maybe you should use
'hive.stats.jdbcdriver="org.apache.mysql.jdbc.EmbeddedDriver"'
settings?

via
http://mail-archives.apache.org/mod_mbox/hive-user/201103.mbox/%[EMAIL PROTECTED]%3E

On Fri, Aug 19, 2011 at 5:45 AM, bharath vissapragada <
[EMAIL PROTECTED]> wrote:

> Hi,
>
> Iam also getting the same error. However I am using mysql for stats.
>
> The thing is I configured mysql for metastore and it works fine and all the
> metadata gets populated normally. When the metastore classes can find the
> mysql jar in the class path , why cant the stats publisher find it. I looked
> at the stats source and everything looks fine.
>
> My conn string is :
> jdbc:mysql://<ip>:3306/TempStatsStore&user=<name>&password=<pwd>.
>
> Am I missing something?
>
> Thanks
>
>
>
>
> On Thu, Aug 18, 2011 at 8:19 AM, wd <[EMAIL PROTECTED]> wrote:
>
>> The error in log is 'java.lang.ClassNotFoundException:
>> org.postgresql.Driver', not can't connect or user name or password error.
>>
>>
>> On Wed, Aug 17, 2011 at 3:53 PM, Jander g <[EMAIL PROTECTED]> wrote:
>>
>>> Hi,wd
>>>
>>> You should configure "hive.stats.dbconnectionstring" as follows.
>>>
>>>
>>> <property>
>>>  <name>hive.stats.dbconnectionstring</name>
>>>
>>>  <value>jdbc:postgresql://localhost/hive_statsdb?createDatabaseIfNotExist=true&user=hive&password=pwd</value>
>>>
>>>  <description>The default connection string for the database that
>>> stores temporary hive statistics.</description>
>>> </property>
>>>
>>> Regards,
>>>
>>> Jander.
>>>
>>>
>>> On Mon, Aug 15, 2011 at 3:09 PM, wd <[EMAIL PROTECTED]> wrote:
>>>
>>>> hi,
>>>>
>>>> I'm try to use postgres as stats database. And made following settings
>>>> in hive-site.xml
>>>>
>>>>
>>>> <property>
>>>>  <name>hive.stats.dbclass</name>
>>>>  <value>jdbc:postgresql</value>
>>>>  <description>The default database that stores temporary hive
>>>> statistics.</description>
>>>> </property>
>>>>
>>>> <property>
>>>>  <name>hive.stats.autogather</name>
>>>>  <value>true</value>
>>>>  <description>A flag to gather statistics automatically during the
>>>> INSERT OVERWRITE command.</description>
>>>> </property>
>>>>
>>>> <property>
>>>>  <name>hive.stats.jdbcdriver</name>
>>>>  <value>org.postgresql.Driver</value>
>>>>  <description>The JDBC driver for the database that stores temporary
>>>> hive statistics.</description>
>>>> </property>
>>>>
>>>> <property>
>>>>  <name>hive.stats.dbconnectionstring</name>
>>>>
>>>>  <value>jdbc:postgresql://localhost/hive_statsdb?createDatabaseIfNotExist=true;user=hive;password=pwd</value>
>>>>  <description>The default connection string for the database that
>>>> stores temporary hive statistics.</description>
>>>> </property>
>>>>
>>>> I use postgres as hive meta database, so there is a
>>>> postgresql-9.0-801.jdbc4.jar file in lib.
>>>>
>>>> After run 'analyse table t1 partitions(dt) comput statistics;' in hive
>>>> cli, it will output some stats info in cli, but nothing in db. And I
>>>> can found there is the flowing errors
>>>>
>>>> 1-08-15 14:54:54,767 INFO
>>>> org.apache.hadoop.hive.ql.exec.TableScanOperator: Stats Gathering
>>>> found a new partition spec = dt=20110805
>>>> 2011-08-15 14:54:54,767 INFO
>>>> org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 forwarding 1 rows
>>>> 2011-08-15 14:54:54,767 INFO ExecMapper: ExecMapper: processing 1
>>>> rows: used memory = 39953640
>>>> 2011-08-15 14:54:54,768 INFO
>>>> org.apache.hadoop.hive.ql.exec.MapOperator: 1 finished. closing...
>>>> 2011-08-15 14:54:54,768 INFO
>>>> org.apache.hadoop.hive.ql.exec.MapOperator: 1 forwarded 2 rows
>>>> 2011-08-15 14:54:54,768 INFO
>>>> org.apache.hadoop.hive.ql.exec.MapOperator: DESERIALIZE_ERRORS:0
>>>> 2011-08-15 14:54:54,768 INFO
>>>> org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 finished.
>>>> closing...
>>>> 2011-08-15 14:54:54,768 INFO
>>>> org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 forwarded 2 rows
>>>> 2011-08-15 14:54:54,772 ERROR
>>>> org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsPublisher: Error during
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB