Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Drill >> mail # dev >> Expressions in the Logical Plan


Copy link to this message
-
Re: Expressions in the Logical Plan
On Nov 17, 2012, at 1:13 PM, Jacques Nadeau <[EMAIL PROTECTED]> wrote:

> Let me give an answer that may help understanding.  The logical plan tries
> to be as generic as possible.  As such, all types are basic types.  That
> being said, we'll surely have a large selection of various types of
> functions which could modify a value.  For example currency(integer) might
> convert a number of pennies into a dollar amount 3500 > "$35.00".  The
> query layer will be responsible for converting the particular query
> language to a set of these various functions.  Then the optimizer decides
> where to apply those functions.  Lastly, the execution layer runs them as
> efficiently as possible givens it execution environment.  Given all of
> this, it is highly likely (especially early on) that a new parser may also
> introduce new functions that need to be supported at the execution layer.

Sorry I've been quiet on the list recently.

I would be delighted to help Drill use the Optiq framework. It has all of this stuff, is proven and extensible.

Specially regarding expressions. Take a look at RexProgram. It models a collection of expressions required to implement a project ("select clause") and a filter. RexProgram is a convenient base for common-subexpression elimination, shortcut generation, and so forth.

Some of the finer details: There is an array of expressions, topologically sorted such that each expression is based only on expressions earlier in the array. Expressions are either inputs, literals, or calls to functions; functions can have multiple arguments, but the arguments must be references to expressions earlier in the array. Expressions are very strongly typed; casting which might be implicit in the query has been made explicit, so arguments have exactly the type required by the function.  Although it is a java class, it would easily serialize to a form similar to a text form similar to the examples in this thread.

https://github.com/julianhyde/optiq/blob/master/src/main/java/org/eigenbase/rex/RexProgram.java

I've been busy on my book and other projects, and I remain too busy to take ownership for of a Drill component, but I would be happy to help someone else evaluate Optiq (or pieces of Optiq) for use in Drill.

Julian
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB