-Re: complex types and ORC
Owen O'Malley 2013-05-03, 21:29
On Mon, Apr 29, 2013 at 4:26 PM, Sean McNamara
> If I create a table that has a map<string, string> field, will ORC files
> columnarize by the keys in the map? Or will all the pairs in the map be
> grouped together?
It will break the map keys into one sub-column and the map values into a
separate sub-column. The map keys will end up in a dictionary and the
values will be in a different dictionary. It would be interesting to have a
map encoding that columnarized low cardinality maps, but I haven't heard
anyone thinking about it.
> My goal is to columnarize the storage of a variable number of fields
> (where the names and number of fields are unknown at schema creation). It
> does not appear to be possible with RCFILE, and I'm curious if ORC just
> adds better type support, or if they also columnarize the fields within
> complex types.