Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive >> mail # user >> Array index support non-constant expresssion


Copy link to this message
-
Array index support non-constant expresssion

Hi, In our project to use the HIVE on CDH3U4 release (Hive 0.7.1), I have a hive table like the following:
Table foo (   search_results array<struct<id:bigint,..............>>   search_clicks array<struct<index_loc:int, ..............>>)
As you can see, the 2nd column, which represents the list of search results clicked, contains the index location of which results was clicked (starting from 0). Now I need to find out the click count of some IDs, but this IDs only exists in the search_result struct.
When I tried to write a query for this, I will try to do following first:

select search_results[c.index_loc] from foo lateral view explode(search_clicks) clickTable as c;
But it won't work in hive, as the following error message returned:AILED: Error in semantic analysis: Line 1:7 Non-constant expressions for array indexes not supported rank.
It looks like that currently Hive (at least 0.7) does NOT support non-constant expressions as the array index.
I searched on google, and found out the following HIVE jira ticket:
https://issues.apache.org/jira/browse/HIVE-1955
It looks like someone has the same request to support it, but not yet.
But there is a comment in the above ticket that it can be done in an UDF. My question is, can anyone share some ideas about how to archive this in an UDF, as it maybe the only option for me right now?
Thanks    
+
java8964 java8964 2012-12-12, 17:15
+
java8964 java8964 2012-12-12, 18:28
+
Navis류승우 2012-12-13, 00:06
+
java8964 java8964 2012-12-13, 01:43
+
Navis류승우 2012-12-13, 04:46