On Nov 17, 2012, at 1:13 PM, Jacques Nadeau <[EMAIL PROTECTED]> wrote:
> Let me give an answer that may help understanding. The logical plan tries
> to be as generic as possible. As such, all types are basic types. That
> being said, we'll surely have a large selection of various types of
> functions which could modify a value. For example currency(integer) might
> convert a number of pennies into a dollar amount 3500 > "$35.00". The
> query layer will be responsible for converting the particular query
> language to a set of these various functions. Then the optimizer decides
> where to apply those functions. Lastly, the execution layer runs them as
> efficiently as possible givens it execution environment. Given all of
> this, it is highly likely (especially early on) that a new parser may also
> introduce new functions that need to be supported at the execution layer.
Sorry I've been quiet on the list recently.
I would be delighted to help Drill use the Optiq framework. It has all of this stuff, is proven and extensible.
Specially regarding expressions. Take a look at RexProgram. It models a collection of expressions required to implement a project ("select clause") and a filter. RexProgram is a convenient base for common-subexpression elimination, shortcut generation, and so forth.
Some of the finer details: There is an array of expressions, topologically sorted such that each expression is based only on expressions earlier in the array. Expressions are either inputs, literals, or calls to functions; functions can have multiple arguments, but the arguments must be references to expressions earlier in the array. Expressions are very strongly typed; casting which might be implicit in the query has been made explicit, so arguments have exactly the type required by the function. Although it is a java class, it would easily serialize to a form similar to a text form similar to the examples in this thread.
I've been busy on my book and other projects, and I remain too busy to take ownership for of a Drill component, but I would be happy to help someone else evaluate Optiq (or pieces of Optiq) for use in Drill.