Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Drill, mail # dev - hangout

Timothy Chen 2013-11-19, 15:52
Michael Hausenblas 2013-11-19, 17:07
Yash Sharma 2013-11-20, 06:17
Timothy Chen 2013-11-20, 13:15
Yash Sharma 2013-11-20, 14:39
Jacques Nadeau 2013-11-20, 14:56
Yash Sharma 2013-11-20, 18:42
Jinfeng Ni 2013-11-20, 22:52
Copy link to this message
Re: hangout
Jinfeng Ni 2013-11-21, 05:12
Hi Yash,

I took a quick look at your github repo. Here are my thoughts.

1. We need implement a cast function.  This cast function will be used 1)
as the explicit cast function used by drill user, 2)to support the implicit
cast used internally within Drill.

For instance, in OperatorFunctionResolver.getBestMatch(), once find the
best matched function implementation, if the parameter type is not same as
argument type, we need inject a cast function as : CAST(
argument_logical_expression   as parameter_type), replace this cast
expression as the argument to the function call.

2. OperatorFunctionResolver.getBestMatch() currently did not check type
match; it simply computes the distance between the param type and argument
type in the TypePrecedeence map, and returns the function implementation
with the smallest total distance (i.e mininum cost).  This seems not right,
since for implicit cast,  at least we need make sure the argument type is
legal to cast to param type.  This is not always true.  For instance, if we
have operator :   DATE op DATE.  Now if we have an expression  float8 op
float 8. Are we implicitly casting from float8 into DATE?  ( Let's say op
is only defined for DATE type, so it's the best match available for
expression float8 op float 8).

Here, I think we probably miss the part to check if one type is CASTABLE
from another type.

3.  For the above reason, internally we need implement an isCastable()
mechanism.  We probably can partly re-use optiq code, which uses a
Map<SqlTypeName, Set<SqlTypeName>> to check if one type is castable to

4. Null expression is castable to any data type. It's right that you put in
NULL as the 1st one in TypePrecedence map.  When function argument(s) is
NULL, once we find the best match function implementation, we could either
replace it with a target_type NULL constant (As Jacques suggested), or
replace NULL with CAST(NULL as target_type) expression.

For instance,  1 + NULL : it matches Int + Int, and NULL is replaced
either int NULL constant, or CAST(NULL as int).

 NULL + NULL : it matches TinyInt + TinyInt (using the getBestMatch
method).  Both NULL operands will be replaced with TinyINT null
or CAST(NULL as TinyInt).

5. Nullable vs Non-nullable. I think implicit cast probably need
support cast from Non-nullable exp

Nullable exp. Otherwise, for each operator and each type, we at least
have to implement 4 versions:

     1) Nullable  int +  Nullable int

     2) Nullable  int + Non-Nullalbe int

     3) Non-Nullable int + Nullable int

     4) Non-Nullable int + Non-Nullalbe int.

By implicit cast Non-nullable exp into Nullable exp, we only need
define one version: Nullable int + Nullable int.
. How to handle the null input could be specified in
function/operator's implementation.

6. Output type. Current code uses OutputTypeDeterminer to decide the
output type. In my view, once we

find the best match function implementation ( based on the argument
types vs parameter types), we would

use the matched function implementations' output type. That is,
function resolver would 1) find the best match

function implementation, 2) decide the output type. These two tasks
will be done in one step.
On Wed, Nov 20, 2013 at 2:52 PM, Jinfeng Ni <[EMAIL PROTECTED]> wrote:

> Hi Yash,
> I'll take a look at your github repo, and start from there.
> To get null expression and pass optiq's parsing, you may try this:
> When the column does not exists, it will return with
> NullExpression.INSTANCE. This way, you can see how the operator "+" is
> binded and the code generator handles the code generation for "+" operator,
> when input  is 1 and a null expression.
> Jinfeng
> s
> On Wed, Nov 20, 2013 at 10:42 AM, Yash Sharma <[EMAIL PROTECTED]>wrote:
>> It would be great working with Jinfeng..
>> For now,
>> I have pushed code for (3) to look for NullExpressions on the same github
>> repo:
>> https://github.com/yashs360/incubator-drill-casting
Yash Sharma 2013-11-21, 14:45
Julian Hyde 2013-11-21, 22:46
Timothy Chen 2014-01-21, 17:07