Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> complex types and ORC


Copy link to this message
-
Re: complex types and ORC
On Mon, Apr 29, 2013 at 4:26 PM, Sean McNamara
<[EMAIL PROTECTED]>wrote:

>  If I create a table that has a map<string, string> field, will ORC files
> columnarize by the keys in the map?  Or will all the pairs in the map be
> grouped together?
>

It will break the map keys into one sub-column and the map values into a
separate sub-column. The map keys will end up in a dictionary and the
values will be in a different dictionary. It would be interesting to have a
map encoding that columnarized low cardinality maps, but I haven't heard
anyone thinking about it.

-- Owen
>  My goal is to columnarize the storage of a variable number of fields
> (where the names and number of fields are unknown at schema creation).  It
> does not appear to be possible with RCFILE, and I'm curious if ORC just
> adds better type support, or if they also columnarize the fields within
> complex types.
>
>  Thanks,
>
>  Sean
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB