Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Drill, mail # dev - Discussion of implicit cast design in drill engine.


Copy link to this message
-
Re: Discussion of implicit cast design in drill engine.
Jacques Nadeau 2013-12-16, 05:57
This looks good.  Do you have a JIRA to attach this work/discussion?

thx,
j
On Mon, Dec 9, 2013 at 10:16 PM, Jinfeng Ni <[EMAIL PROTECTED]> wrote:

> Hi Yash / All,
>
> I gave some thoughts about how to implement implicit cast. Here is my
> preliminary thoughts.
>
> 1 .  implicit cast vs explicit cast.
>
> Basically, implicit cast would need leverage explicit cast's
> implementation. For instance, given an logical expression :  1 + 3.0, if
> the function resolver finds that argument 1 should be implicitly casted
> into float4, then, drill code should transform the logical expression into
>  cast(1 as float4) + 3.0, so that "+" operator will call the add(float4,
> float4) implementation.
>
> 2. Add implicit cast function call in the logical expression tree.
>
> Currently, drill code uses
> FunctionImplementationRegistry.getFunction(FunctionCall)
> to get DrillFuncHolder in EvalVisitor.
>
> Your FunctionResolver is used to find the best match in the call of
> getFunction(). However, if the best match says implicit cast is required,
> it's kind of difficult to let getFunction() do 1) insert the cast function
> to the logical expression tree, and 2) get cast's DrillFuncHolder, and 3)
> generate the code for the cast function.
>
> We probably should separate the process of adding implicit cast from the
> logic of FunctionImplementationRegistry.getFunction() and code generation.
>
> To do that :
>
>    -      Introduce a new Visitor class ImplicitCastBuilder.
>    -      ImplicitCastBuilder will look similar to EvalVisitor .
>     ImplicitCastBuilder extends AbstractExprVisitor<*LogicalExpression*,
>    CodeGenerator<?>, RuntimeException>
>    -     ImplicitCastBuilder should build cast function call in *bottom-up*
>     way.
>    -     ImplicitCastBuilder will modify logical expression tree, and
>    replace an argument with a cast function call, if your
>    *FunctionResolver.getBestMatch() *shows implicit cast is required.
>    -    CodeGenerator.addExpr will call ImplicitCastBuilder to insert the
>    implicit cast() to logical expression tree .
>
>
>   public HoldingContainer addExpr(LogicalExpression ex, boolean rotate){
>
> //    logger.debug("Adding next write {}", ex);
>
>     if(rotate) rotateBlock();
>
>     *ex = implicitCastBuilder.accept(ex, this);*
>
>     return evaluationVisitor.addExpr(ex, this);
>
>   }
>
>
>    -    EvalVisitor will then do the match() and code generation as before.
>    ( No need to call your *FunctionResolver.getBestMatch() *at this stage,
>    since all the required implicit cast has been inserted into the logical
>    expression tree).
>
>
> For example, let's say we have logical expression tree f1 ( a1, f2( a2,
> a3))
>
>                                f1()
>                             /      \
>                           /         \
>                           a1       f2()
>                                    /     \
>                                  /        \
>                                a2       a3
>
> We may end up with the following logical expression tree, after
> ImplicitCastBuilder visit.
>
>                               f1()
>                             /      \
>                           /         \
>                           a1     cast1()
>                                        \
>                                         \
>                                           f2()
>                                          /   \
>                                        /      \
>                                      a2     cast2()
>                                                 |
>                                                 |
>                                                a3
>
> Note that cast2 would be inserted first, followed by cast1 during the
> visite, since we need do it in bottom-up ( when we do getBestMatch() for f1
> and insert cast1, we need know the output type of cast2, in order to
> determine output type of f2(), which is argument to f1() ).
>
> Please let me know your thoughts. Thanks!