Dear Hive User List,

 

I am trying to build indexes on a hive 0.7.1 environment using postgresql as
the metastore, but it is failing silently.

 

The following command doesn't generate any errors:

 

hive> CREATE TABLE t (i INT);

OK

Time taken: 0.299 seconds

hive> CREATE INDEX i ON TABLE t (i) with 'COMPACT' WITH DEFERRED REBUILD;

OK

Time taken: 0.287 seconds

 

A directory is created for the index in  "/usr/hive/warehouse/", but the
index can't be found:

 

hive> show tables;

OK

t

Time taken: 0.163 seconds

hive> show indexes on t;

OK

Time taken: 0.303 seconds

 

There are no errors in the hive.log file, and I am running the 0.7.1
release.

 

Thank you,

Clint

 

--

The information contained in this message may be privileged and/or
confidential and protected from disclosure. If the reader of this message is
not the intended recipient or an employee or agent responsible for
delivering this message to the intended recipient, you are hereby notified
that any dissemination, distribution or copying of this communication is
strictly prohibited. If you have received this communication in error,
please notify the sender immediately by replying to this message and
deleting the material from any computer.

 

 

Hi Clint,

Indeed this is a bug, "DEFERRED_REBUILD" should be boolean and not bit(1) in
"IDXS".

Regards,
Esteban.

--
Support Engineer, Cloudera.
On Wed, Jun 22, 2011 at 11:25 AM, Clint Green <[EMAIL PROTECTED]>wrote:

> Dear Hive User List,****
>
> ** **
>
> I am trying to build indexes on a hive 0.7.1 environment using postgresql
> as the metastore, but it is failing silently.****
>
> ** **
>
> The following command doesn’t generate any errors:****
>
> ** **
>
> hive> CREATE TABLE t (i INT);****
>
> OK****
>
> Time taken: 0.299 seconds****
>
> hive> CREATE INDEX i ON TABLE t (i) with ‘COMPACT’ WITH DEFERRED REBUILD;*
> ***
>
> OK****
>
> Time taken: 0.287 seconds****
>
> ** **
>
> A directory is created for the index in  “/usr/hive/warehouse/”, but the
> index can’t be found:****
>
> ** **
>
> hive> show tables;****
>
> OK****
>
> t****
>
> Time taken: 0.163 seconds****
>
> hive> show indexes on t;****
>
> OK****
>
> Time taken: 0.303 seconds****
>
> ** **
>
> There are no errors in the hive.log file, and I am running the 0.7.1
> release.****
>
> ** **
>
> Thank you,****
>
> Clint****
>
> ** **
>
> --****
>
> The information contained in this message may be privileged and/or
> confidential and protected from disclosure. If the reader of this message is
> not the intended recipient or an employee or agent responsible for
> delivering this message to the intended recipient, you are hereby notified
> that any dissemination, distribution or copying of this communication is
> strictly prohibited. If you have received this communication in error,
> please notify the sender immediately by replying to this message and
> deleting the material from any computer. ****
>
> ** **
>
> ** **
>

Clint,

sorry, I think I was too cryptic :)

The default schema creation script is setting the column type for
"DEFERRED_REBUILD"
as bit(1) instead of boolean in the "IDXS" table and the JDBC driver is
failing silently for that type mismatch. Also, it seems that "IS_COMPRESSED"
in one of the internal datanucleus tables has the same issue in 0.7.1. A
temporary solution is to alter the bit(1) type on those columns and set them
to boolean, that should work for you.

Cheers,
Esteban.

--
Support Engineer, Cloudera.
On Wed, Jun 22, 2011 at 12:04 PM, Esteban Gutierrez <[EMAIL PROTECTED]>wrote:

>
>
> Hi Clint,
>
> Indeed this is a bug, "DEFERRED_REBUILD" should be boolean and not bit(1)
> in "IDXS".
>
> Regards,
> Esteban.
>
> --
> Support Engineer, Cloudera.
>
>
>
>
> On Wed, Jun 22, 2011 at 11:25 AM, Clint Green <[EMAIL PROTECTED]>wrote:
>
>> Dear Hive User List,****
>>
>> ** **
>>
>> I am trying to build indexes on a hive 0.7.1 environment using postgresql
>> as the metastore, but it is failing silently.****
>>
>> ** **
>>
>> The following command doesn’t generate any errors:****
>>
>> ** **
>>
>> hive> CREATE TABLE t (i INT);****
>>
>> OK****
>>
>> Time taken: 0.299 seconds****
>>
>> hive> CREATE INDEX i ON TABLE t (i) with ‘COMPACT’ WITH DEFERRED REBUILD;
>> ****
>>
>> OK****
>>
>> Time taken: 0.287 seconds****
>>
>> ** **
>>
>> A directory is created for the index in  “/usr/hive/warehouse/”, but the
>> index can’t be found:****
>>
>> ** **
>>
>> hive> show tables;****
>>
>> OK****
>>
>> t****
>>
>> Time taken: 0.163 seconds****
>>
>> hive> show indexes on t;****
>>
>> OK****
>>
>> Time taken: 0.303 seconds****
>>
>> ** **
>>
>> There are no errors in the hive.log file, and I am running the 0.7.1
>> release.****
>>
>> ** **
>>
>> Thank you,****
>>
>> Clint****
>>
>> ** **
>>
>> --****
>>
>> The information contained in this message may be privileged and/or
>> confidential and protected from disclosure. If the reader of this message is
>> not the intended recipient or an employee or agent responsible for
>> delivering this message to the intended recipient, you are hereby notified
>> that any dissemination, distribution or copying of this communication is
>> strictly prohibited. If you have received this communication in error,
>> please notify the sender immediately by replying to this message and
>> deleting the material from any computer. ****
>>
>> ** **
>>
>> ** **
>>
>
>

Esteban,

 

Thanks for the quick reply!  That resolved the issue.  

 

The "IS_COMPRESSED" error is also in the HIVE-2176.3.patch.

 

Clint

 

From: Esteban Gutierrez [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, June 22, 2011 3:14 PM
To: [EMAIL PROTECTED]
Subject: Re: Trouble creating indexes with psql metastore

 

 

Clint,

 

sorry, I think I was too cryptic :)

 

The default schema creation script is setting the column type for
"DEFERRED_REBUILD" as bit(1) instead of boolean in the "IDXS" table and the
JDBC driver is failing silently for that type mismatch. Also, it seems that
"IS_COMPRESSED" in one of the internal datanucleus tables has the same issue
in 0.7.1. A temporary solution is to alter the bit(1) type on those columns
and set them to boolean, that should work for you.

 

Cheers,
Esteban.

 

--

Support Engineer, Cloudera.

 

On Wed, Jun 22, 2011 at 12:04 PM, Esteban Gutierrez <[EMAIL PROTECTED]>
wrote:

 

 

Hi Clint,

 

Indeed this is a bug, "DEFERRED_REBUILD" should be boolean and not bit(1) in
"IDXS".

 

Regards,

Esteban.

 

--

Support Engineer, Cloudera.

 

On Wed, Jun 22, 2011 at 11:25 AM, Clint Green <[EMAIL PROTECTED]>
wrote:

Dear Hive User List,

 

I am trying to build indexes on a hive 0.7.1 environment using postgresql as
the metastore, but it is failing silently.

 

The following command doesn't generate any errors:

 

hive> CREATE TABLE t (i INT);

OK

Time taken: 0.299 seconds

hive> CREATE INDEX i ON TABLE t (i) with 'COMPACT' WITH DEFERRED REBUILD;

OK

Time taken: 0.287 seconds

 

A directory is created for the index in  "/usr/hive/warehouse/", but the
index can't be found:

 

hive> show tables;

OK

t

Time taken: 0.163 seconds

hive> show indexes on t;

OK

Time taken: 0.303 seconds

 

There are no errors in the hive.log file, and I am running the 0.7.1
release.

 

Thank you,

Clint

 

--

The information contained in this message may be privileged and/or
confidential and protected from disclosure. If the reader of this message is
not the intended recipient or an employee or agent responsible for
delivering this message to the intended recipient, you are hereby notified
that any dissemination, distribution or copying of this communication is
strictly prohibited. If you have received this communication in error,
please notify the sender immediately by replying to this message and
deleting the material from any computer.

 

 

 

 

I have a similar problem with a trunk build and a mysql metastore.
Doing: alter table IDXS modify column DEFERRED_REBUILD boolean not null;
Doesn't seem to fix it. Perhaps because mysql converts the boolean into
a "tinyint(1)"?

Is there an easy way to make it fail with an error instead of getting an OK?
On 06/22/2011 09:04 PM, Esteban Gutierrez wrote:
>
>
> Hi Clint,
>
> Indeed this is a bug, "DEFERRED_REBUILD" should be boolean and not
> bit(1) in "IDXS".
>
> Regards,
> Esteban.
>
> --
> Support Engineer, Cloudera.
>
>
>
>
> On Wed, Jun 22, 2011 at 11:25 AM, Clint Green <[EMAIL PROTECTED]
> <mailto:[EMAIL PROTECTED]>> wrote:
>
>     Dear Hive User List,
>
>     I am trying to build indexes on a hive 0.7.1 environment using
>     postgresql as the metastore, but it is failing silently.
>
>     The following command doesn�t generate any errors:
>
>     hive> CREATE TABLE t (i INT);
>
>     OK
>
>     Time taken: 0.299 seconds
>
>     hive> CREATE INDEX i ON TABLE t (i) with �COMPACT� WITH DEFERRED
>     REBUILD;
>
>     OK
>
>     Time taken: 0.287 seconds
>
>     A directory is created for the index in  ï¿½/usr/hive/warehouse/�,
>     but the index can�t be found:
>
>     hive> show tables;
>
>     OK
>
>     t
>
>     Time taken: 0.163 seconds
>
>     hive> show indexes on t;
>
>     OK
>
>     Time taken: 0.303 seconds
>
>     There are no errors in the hive.log file, and I am running the
>     0.7.1 release.
>
>     Thank you,
>
>     Clint
>
>     --
>
>     The information contained in this message may be privileged and/or
>     confidential and protected from disclosure. If the reader of this
>     message is not the intended recipient or an employee or agent
>     responsible for delivering this message to the intended recipient,
>     you are hereby notified that any dissemination, distribution or
>     copying of this communication is strictly prohibited. If you have
>     received this communication in error, please notify the sender
>     immediately by replying to this message and deleting the material
>     from any computer.
>
>

Dear Hive User List,

 

I am having a new issue with creating indexes when using the $ hive -service
metastore option.

 

When I do so from a client, I get a null pointer exception and cannot run
any other commands until I close and reopen the Hive CLI.  The metastore log
states "String index out of range exception."

 

If I create the same index from a client directly connecting to the postgres
metastore, it succeeds.

 

I am running Hive 0.7.1, having upgraded from 0.6.0 -> 0.7.0 (with postgres
metastore scripts) -> 0.7.1 (with "IS_COMPRESSED" and "DEFERRED_REBUILD"
fixes).

 

Thank you in advance for all your help (again!).

 

Clint

 

--

The information contained in this message may be privileged and/or
confidential and protected from disclosure. If the reader of this message is
not the intended recipient or an employee or agent responsible for
delivering this message to the intended recipient, you are hereby notified
that any dissemination, distribution or copying of this communication is
strictly prohibited. If you have received this communication in error,
please notify the sender immediately by replying to this message and
deleting the material from any computer.