Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Drill >> mail # dev >> Discussion of implicit cast design in drill engine.


Copy link to this message
-
Re: Discussion of implicit cast design in drill engine.
This looks good.  Do you have a JIRA to attach this work/discussion?

thx,
j
On Mon, Dec 9, 2013 at 10:16 PM, Jinfeng Ni <[EMAIL PROTECTED]> wrote:

> Hi Yash / All,
>
> I gave some thoughts about how to implement implicit cast. Here is my
> preliminary thoughts.
>
> 1 .  implicit cast vs explicit cast.
>
> Basically, implicit cast would need leverage explicit cast's
> implementation. For instance, given an logical expression :  1 + 3.0, if
> the function resolver finds that argument 1 should be implicitly casted
> into float4, then, drill code should transform the logical expression into
>  cast(1 as float4) + 3.0, so that "+" operator will call the add(float4,
> float4) implementation.
>
> 2. Add implicit cast function call in the logical expression tree.
>
> Currently, drill code uses
> FunctionImplementationRegistry.getFunction(FunctionCall)
> to get DrillFuncHolder in EvalVisitor.
>
> Your FunctionResolver is used to find the best match in the call of
> getFunction(). However, if the best match says implicit cast is required,
> it's kind of difficult to let getFunction() do 1) insert the cast function
> to the logical expression tree, and 2) get cast's DrillFuncHolder, and 3)
> generate the code for the cast function.
>
> We probably should separate the process of adding implicit cast from the
> logic of FunctionImplementationRegistry.getFunction() and code generation.
>
> To do that :
>
>    -      Introduce a new Visitor class ImplicitCastBuilder.
>    -      ImplicitCastBuilder will look similar to EvalVisitor .
>     ImplicitCastBuilder extends AbstractExprVisitor<*LogicalExpression*,
>    CodeGenerator<?>, RuntimeException>
>    -     ImplicitCastBuilder should build cast function call in *bottom-up*
>     way.
>    -     ImplicitCastBuilder will modify logical expression tree, and
>    replace an argument with a cast function call, if your
>    *FunctionResolver.getBestMatch() *shows implicit cast is required.
>    -    CodeGenerator.addExpr will call ImplicitCastBuilder to insert the
>    implicit cast() to logical expression tree .
>
>
>   public HoldingContainer addExpr(LogicalExpression ex, boolean rotate){
>
> //    logger.debug("Adding next write {}", ex);
>
>     if(rotate) rotateBlock();
>
>     *ex = implicitCastBuilder.accept(ex, this);*
>
>     return evaluationVisitor.addExpr(ex, this);
>
>   }
>
>
>    -    EvalVisitor will then do the match() and code generation as before.
>    ( No need to call your *FunctionResolver.getBestMatch() *at this stage,
>    since all the required implicit cast has been inserted into the logical
>    expression tree).
>
>
> For example, let's say we have logical expression tree f1 ( a1, f2( a2,
> a3))
>
>                                f1()
>                             /      \
>                           /         \
>                           a1       f2()
>                                    /     \
>                                  /        \
>                                a2       a3
>
> We may end up with the following logical expression tree, after
> ImplicitCastBuilder visit.
>
>                               f1()
>                             /      \
>                           /         \
>                           a1     cast1()
>                                        \
>                                         \
>                                           f2()
>                                          /   \
>                                        /      \
>                                      a2     cast2()
>                                                 |
>                                                 |
>                                                a3
>
> Note that cast2 would be inserted first, followed by cast1 during the
> visite, since we need do it in bottom-up ( when we do getBestMatch() for f1
> and insert cast1, we need know the output type of cast2, in order to
> determine output type of f2(), which is argument to f1() ).
>
> Please let me know your thoughts. Thanks!
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB