Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Drill, mail # dev - [2/6] git commit: DRILL-316: Implement explicit casting Includes following changes:


Copy link to this message
-
[2/6] git commit: DRILL-316: Implement explicit casting Includes following changes:
jacques@... 2014-01-14, 16:48
DRILL-316: Implement explicit casting
Includes following changes:

Parse syntax to support explicit cast function in physical or logical plan.

prototype of explicit casting : parsing from plan's expr, and call generated cast code.

Prototype of explicit cast with unit test.

explicit cast: create different cast for type with fixed size vs with var size.

explicit cast : minor change in comments, varname, etc.

Drill-316 : explicit cast. change func def for cast with var size.

Drill-316: minor change in unit test case for explicit cast.

Drill-316 : modify the function def for cast functions.

DRILL-316: prototype impl of cast from/to var-length type.

Dirll-316 : add test case for explicit cast.

Drill-316: explicit cast into varchar type from int, bigint, float4, float8. unit test physical plan added.

Drill-316: change parser syntax for explicit cast. Add more test case physical plans.

DRILL-316: explict cast. Use functiom template to generate all cast functions. Currently, excplict
cast supports cast between int, bigint, float4, float8, varchar, varbinary. (total 30 combinations).

Drill-316: explict cast - add test case for nested cast functions.

Drill-316: minor change to function template for explicit cast.

DRILL-316: add junit test case for explict cast, using 7 json physical plans.

DRILL-316: explict cast. remove a test json file.

DRILL-316: explicit cast. add a json testcase.

DRILL-316: check return type for function resolution. delete two un-needed test json files.

Drill-316: code change for review comment 1 and 2.

Drill-316: address review comment 3.

Drill-316: address testcase comments.

Drill-316: Decode byte array directly into int/long, when cast varchar/varbinary into int/bigint.

Drill-316: minor change to grammer rule for cast.
Project: http://git-wip-us.apache.org/repos/asf/incubator-drill/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-drill/commit/931b641b
Tree: http://git-wip-us.apache.org/repos/asf/incubator-drill/tree/931b641b
Diff: http://git-wip-us.apache.org/repos/asf/incubator-drill/diff/931b641b

Branch: refs/heads/master
Commit: 931b641b1dfecdc5d1c0b65ebfbff6bbda099649
Parents: f95e933
Author: Jinfeng Ni <[EMAIL PROTECTED]>
Authored: Fri Nov 22 17:56:56 2013 -0800
Committer: Jacques Nadeau <[EMAIL PROTECTED]>
Committed: Tue Jan 14 08:40:41 2014 -0800

----------------------------------------------------------------------
 .../drill/common/expression/parser/ExprLexer.g  |  11 +
 .../drill/common/expression/parser/ExprParser.g |  40 ++
 .../drill/common/expression/FunctionCall.java   |   7 +-
 .../common/expression/FunctionDefinition.java   |   4 +
 .../common/expression/FunctionRegistry.java     |  29 +-
 .../common/expression/OutputTypeDeterminer.java |   8 +-
 .../common/expression/fn/CastFunctionDefs.java  |  38 ++
 exec/java-exec/src/main/codegen/data/Casts.tdd  |  44 ++-
 .../main/codegen/templates/CastFunctions.java   |   8 +-
 .../templates/CastFunctionsSrcVarLen.java       | 131 +++++++
 .../CastFunctionsSrcVarLenTargetVarLen.java     |  69 ++++
 .../templates/CastFunctionsTargetVarLen.java    |  67 ++++
 .../drill/exec/expr/fn/DrillFuncHolder.java     |   2 +
 .../expr/fn/FunctionImplementationRegistry.java |   1 +
 .../drill/exec/vector/ValueHolderHelper.java    |   2 +-
 .../exec/physical/impl/TestCastFunctions.java   | 378 +++++++++++++++++++
 .../functions/cast/testCastBigInt.json          |  43 +++
 .../functions/cast/testCastFloat4.json          |  44 +++
 .../functions/cast/testCastFloat8.json          |  44 +++
 .../resources/functions/cast/testCastInt.json   |  43 +++
 .../functions/cast/testCastNested.json          |  43 +++
 .../functions/cast/testCastNumException.json    |  37 ++
 .../functions/cast/testCastVarBinary.json       |  55 +++
 .../functions/cast/testCastVarChar.json         |  51 +++
 .../drill/jdbc/test/FullEngineTPCHTest.java     | 127 -------
 25 files changed, 1177 insertions(+), 149 deletions(-)
---------------------------------------

http://git-wip-us.apache.org/repos/asf/incubator-drill/blob/931b641b/common/src/main/antlr3/org/apache/drill/common/expression/parser/ExprLexer.g
diff --git a/common/src/main/antlr3/org/apache/drill/common/expression/parser/ExprLexer.g b/common/src/main/antlr3/org/apache/drill/common/expression/parser/ExprLexer.g
index 76da965..be2a3f2 100644
+++ b/common/src/main/antlr3/org/apache/drill/common/expression/parser/ExprLexer.g
@@ -35,6 +35,17 @@ In       : 'in';
 Case     : 'case';
 When     : 'when';
 
+Cast: 'cast';
+Nullable: 'nullable';
+Repeat: 'repeat';
+As: 'as';
+
+INT : 'int';
+BIGINT : 'bigint';
+FLOAT4   : 'float4';
+FLOAT8   : 'float8';
+VARCHAR  : 'varchar';
+VARBINARY: 'varbinary';
 
 Or       : '||' | 'or' | 'OR' | 'Or';
 And      : '&&' | 'and' | 'AND' ;

http://git-wip-us.apache.org/repos/asf/incubator-drill/blob/931b641b/common/src/main/antlr3/org/apache/drill/common/expression/parser/ExprParser.g
diff --git a/common/src/main/antlr3/org/apache/drill/common/expression/parser/ExprParser.g b/common/src/main/antlr3/org/apache/drill/common/expression/parser/ExprParser.g
index 7fc1651..b5cf292 100644
+++ b/common/src/main/antlr3/org/apache/drill/common/expression/parser/ExprParser.g
@@ -35,6 +35,8 @@ package org.apache.drill.common.expression.parser;
 import org.antlr.runtime.BitSet;
 import java.util.*;
 import org.apache.drill.common.expression.*;
+import org.apache.drill.common.types.*;
+import org.apache.drill.common.types.TypeProtos.*;
 
 }
 
@@ -66,7 +68,44 @@ parse returns [LogicalExpression e]
 functionCall returns [LogicalExpression e]
   :  Identifier OParen exprList? CParen {$e = registry.createExpression($Identifier.text, pos($Identifier), $exprList.listE);  }
   ;
+  
+castCall returns [LogicalExpression e]
+ @init{
+    List<LogicalExpression> exprs = new ArrayList<LogicalExpression>();
+  ExpressionPosition p = null;
+ }  
+  :  Cast OParen expression As dataType repeat? CParen
+      {  if ($repeat.isRep!=null && $repeat.isRep.compar