Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive >> mail # user >> UDF for transforming a collection of arrays into an array of structs


Copy link to this message
-
UDF for transforming a collection of arrays into an array of structs
Hi,
I am working on ingesting some legacy data that is denormalized in hive
somewhat like the following:

CREATE TABLE mytable (
           order_id int,

           product_id array<int>,
           product_name array<string>,
           product_price array<bigint>
....
)
As you see, the product_* fields would better be represented as a
struct<int,string,bigint>.
Is there an UDF that can take a group of arrays and, assuming they're all
the same size, return an array of structs instead ?

I know it's not difficult to implement using generic UDFs, but I was
wondering if anybody had already done it, and if not, if anybody was
actually interested in something like that.

R.

--
----------------------------------------------------------
Good judgement comes with experience.
Experience comes with bad judgement.
----------------------------------------------------------
Roberto Congiu - Data Engineer - OpenX
tel: +1 626 466 1141
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB