Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Drill, mail # dev - Review Request 16490: DRILL_259 : implicit cast support in drill.

Jinfeng Ni 2013-12-27, 21:42
Copy link to this message
Re: Review Request 16490: DRILL_259 : implicit cast support in drill.
Jinfeng Ni 2013-12-31, 01:36

This is an automatically generated e-mail. To reply, visit:

(Updated Dec. 30, 2013, 5:36 p.m.)
Review request for drill.

Add code to handle NullExpression : isnull(unknownCol) should return true.

Internally, unknowCol will be represented by NullExpression.INSTANCE. When the code is doing function resolution, if 1) the input expression is NullExpression.INSTANCE and 2)the function implementation allows null as input, then, the code will replace the NullExpression.INSTANCE with a TypedNullConstant. This TypedNullConstant will ensure the corresponding ValueHolder's isSet=0.

New file or modified file:
1. TypedNullConstant.java : to represent a null constant for one particular type.
2. EvaluationVisitor : when see TypedNullConstant, generate the declare block for the null constant.
3. ImplicitCastBuilder : replace NullExpression.INSTANCE with TypedNullConstant when necessary

Unit test: testICastNullExp.json.

  isnull(unknownCol), isnotnull(unknowCol),  1 + unknowCol, 1.2 + unknowCol, nested with explicit cast function, etc.
Repository: drill-git

This patch is to provide implicit cast support in Drill.

The code change is mainly in two areas:

1. Function Resolver and related type cast rules : used to choose the best matched function implementation corresponding to a function name and its list of argument types. Type promotion would be implied, if there is no function implementation whose parameter types exactly match the argument types. Type promotion is implemented using the type precedence list. Type promotion is checked against a rule which will help implement a isCastable(from, to) method.  
New files: 1) DefaultFunctionResolver.java
           2) FunctionResolver
           3) FunctionResolverFactor.java
           4) OperatorFunctionResolver.java : reserved for future use.
           5) ResolverTypePrecedence.java
           6) TypeCastRules.java
           7) UDFFunctionResolver.java : reserved for future use.

2. ImplicitCastBuilder : to inject implicit cast whenever necessary. After the function resolver returns a best matched function implementation, if the param type does not match the argument type, an implicit cast will be inserted on top of the argument.
   - ImplicitCastBuilder uses a visitor to inject implicit cast.
   - ImplicitCastBuilder will be part of ExpressionTreeMateralizer, after we materialize the expression tree.
   - ExpressionValidator will be called after implicit cast is injected, in stead of after expression materialization.
   - Because ImplicitCastBuilder requires a FunctionImplementationRegistry (to decide the best matched function implementation), and ImplicitCastBuilder is part of ExpressionTreeMaterializer, all the call of ExpressionTreeMaterializer is modified by adding a FunctionImplementationRegistry.

New files or modified files:
            4)DrillFuncHolder.java : expose param type and size
            5)FunctionImplementationRegistry.java : expose the list of function implementation.
            6) other files because of the change to ExpressionTreeMaterializer.java.

Some other minor changes:

1. ArgumentValidator.ComparableArguments :
  change allSame from true to false, since currently we have comparison operators defined over different types of inputs.

2. VectorUtil:
  Add code to handle null value when display the result.
Diffs (updated)

  common/src/main/java/org/apache/drill/common/expression/ArgumentValidators.java cd30a06
  common/src/main/java/org/apache/drill/common/expression/TypedNullConstant.java PRE-CREATION
  common/src/main/java/org/apache/drill/common/expression/fn/CastFunctionDefs.java PRE-CREATION
  exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java 5336c0e
  exec/java-exec/src/main/java/org/apache/drill/exec/expr/EvaluationVisitor.java 587fe3c
  exec/java-exec/src/main/java/org/apache/drill/exec/expr/ExpressionTreeMaterializer.java 5d576b1
  exec/java-exec/src/main/java/org/apache/drill/exec/expr/ImplicitCastBuilder.java PRE-CREATION
  exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/DrillFuncHolder.java e001ffe
  exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/FunctionImplementationRegistry.java 5bbab76
  exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/MathFunctions.java 288760b
  exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/AggBatch.java f652e35
  exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/filter/FilterRecordBatch.java fe298d6
  exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/MergeJoinBatch.java f3a32cd
  exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/mergereceiver/MergingRecordBatch.java fd392a3
  exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/orderedpartitioner/OrderedPartitionRecordBatch.java a3d1d09
  exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/partitionsender/PartitionSenderRootExec.java bc53bd2
  exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/project/ProjectRecordBatch.java 41a4c4d
  exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/sort/SortBatch.java 7881115
  exec/java-exec/src/main/java/org/apache/drill/exec/resolver/DefaultFunctionResolver.java PRE-CREATION
  exec/java-exec/src/main/java/org/apache/drill/exec/resolver/FunctionResolver.java PRE-CREATION
  exec/java-exec/src/main/java/org/apache/drill/exec/resolver/FunctionResolverFactory.java PRE-CREATION
  exec/java-exec/src/main/java/org/apache/drill/exec/resolver/OperatorFunctionResolver.java PRE-CREATION
Timothy Chen 2014-01-03, 18:56
Jinfeng Ni 2014-01-03, 19:46
Jinfeng Ni 2014-01-03, 22:37
Yash Sharma 2014-01-04, 16:17
Jinfeng Ni 2014-01-03, 22:58
Jason Altekruse 2014-01-04, 00:20
Jason Altekruse 2014-01-04, 18:02
Jason Altekruse 2014-01-04, 17:48
Jacques Nadeau 2014-01-14, 16:51