Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # user - Hive index error


Copy link to this message
-
Re: Hive index error
Stephen Sprague 2013-08-02, 15:02
and there you go.  you can't use partitioned columns in the index. why?  if
you do a hdfs dfs -ls on the path to your table you'll get some insight on
how hive uses the partitioned columns to organize the table data.   then
you'll see why partitioned columns are quite "different" than the other
columns. :)
On Thu, Aug 1, 2013 at 9:19 PM, Omkar Joshi <[EMAIL PROTECTED]>wrote:

> Hi Mark,****
>
> ** **
>
> I’m using single node Hive installation with all settings default. Yes,
> the table is partitioned on the column comp_code.****
>
> ** **
>
> Regards,****
>
> Omkar Joshi****
>
> ** **
>
> *From:* Mark Wagner [mailto:[EMAIL PROTECTED]]
> *Sent:* Thursday, August 01, 2013 11:15 PM
> *To:* [EMAIL PROTECTED]
> *Subject:* Re: Hive index error****
>
> ** **
>
> Hi Omkar,****
>
> ** **
>
> What Serde does the table use? Given that you're able to index on leg_id
> it doesn't sound quite the same as the issue I hit. Is there any
> partitioning of the table?****
>
> ** **
>
> -Mark****
>
> ** **
>
> On Wed, Jul 31, 2013 at 10:39 PM, Omkar Joshi <[EMAIL PROTECTED]>
> wrote:****
>
> I'm facing issues while building an index on multiple columns in a
> Hive(0.9.0) table.****
>
> describe nas_comps;****
>
> OK****
>
> leg_id  int****
>
> ds_name string****
>
> dep_date        string****
>
> crr_code        string****
>
> flight_no       string****
>
> orgn    string****
>
> dstn    string****
>
> physical_cap    int****
>
> adjusted_cap    int****
>
> closed_cap      int****
>
> comp_code       string****
>
> This works :****
>
> CREATE INDEX nas_comps_legid ON TABLE nas_comps (leg_id) AS 'org.apache.hadoop.hive.ql.index.compact.CompactIndexHandler' WITH DEFERRED REBUILD;****
>
> But this doesn't :****
>
>     CREATE INDEX nas_comps_legid_compcode ON TABLE nas_comps (leg_id,comp_code) AS 'org.apache.hadoop.hive.ql.index.compact.CompactIndexHandler' WITH DEFERRED REBUILD;****
>
>  ****
>
> FAILED: Error in metadata: java.lang.RuntimeException: Check the index columns, they should appear in the table being indexed.****
>
> FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask****
>
> I guess the index is somehow able to recognize only the first column
> because even this failed :****
>
> CREATE INDEX nas_comps_compcode ON TABLE nas_comps (comp_code) AS 'org.apache.hadoop.hive.ql.index.compact.CompactIndexHandler' WITH DEFERRED REBUILD;****
>
>  ****
>
> FAILED: Error in metadata: java.lang.RuntimeException: Check the index columns, they should appear in the table being indexed.****
>
> FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask****
>
> I checked this <https://issues.apache.org/jira/browse/HIVE-4251> issue
> but I don't think this is the cause.****
>
>  ****
>
>  ****
>
> Regards,****
>
> Omkar Joshi****
>
> ** **
> ------------------------------
>
> The contents of this e-mail and any attachment(s) may contain confidential
> or privileged information for the intended recipient(s). Unintended
> recipients are prohibited from taking action on the basis of information in
> this e-mail and using or disseminating the information, and must notify the
> sender and delete it from their system. L&T Infotech will not accept
> responsibility or liability for the accuracy or completeness of, or the
> presence of any virus or disabling code in this e-mail"****
>
> ** **
>