Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # dev >> Review Request: Column Column, and Column Scalar vectorized execution tests

Copy link to this message
Re: Review Request: Column Column, and Column Scalar vectorized execution tests

> On May 14, 2013, 6:13 p.m., Eric Hanson wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/TestColumnScalarOperationVectorExpressionEvaluation.txt, line 1
> > <https://reviews.apache.org/r/11133/diff/2/?file=291141#file291141line1>
> >
> >     test .txt templates need apache license

the generated class will have the apache license (see testclass.txt) but if I add it here then every test case would have it, which would bloat the class with tons of comments. is it necessary? or is there another way to specify it? like maybe a license.txt in the directory?
- tony
This is an automatically generated e-mail. To reply, visit:
On May 14, 2013, 12:34 a.m., tony murphy wrote:
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11133/
> -----------------------------------------------------------
> (Updated May 14, 2013, 12:34 a.m.)
> Review request for hive, Jitendra Pandey, Eric Hanson, Sarvesh Sakalanaga, and Remus Rusanu.
> Description
> -------
> This patch adds Column Column, and Column Scalar vectorized execution tests. These tests are generated in parallel with the vectorized expressions. The tests focus is on validating the column vector and the vectorized row batch metadata regarding nulls, repeating, and selection.
> Overview of Changes:
> CodeGen.java:
> + joinPath, getCamelCaseType, readFile and writeFile made static for use in TestCodeGen.java.
> + filter types now specify null as their output type rather than "doesn't matter" to make detection for test generation easier.
> + support for test generation added.
> TestCodeGen.java & Templates:
>      TestClass.txt
>      TestColumnColumnFilterVectorExpressionEvaluation.txt,
>      TestColumnColumnOperationVectorExpressionEvaluation.txt,
>      TestColumnScalarFilterVectorExpressionEvaluation.txt,
>      TestColumnScalarOperationVectorExpressionEvaluation.txt
> +This class is mutable and maintains a hashmap of TestSuiteClassName to test cases. The tests cases are added over the course of vectorized expressions class generation, with test classes being outputted at the end. For each column vector (inputs and/or outputs) a matrix of pairwise covering Booleans is used to generate test cases across nulls and repeating dimensions. Based on the input column vector(s) nulls and repeating states the states of the output column vector (if there is one) is validated, along with the null vector. For filter operations the selection vector is validated against the generated data. Each template corresponds to a class representing a test suite.
> VectorizedRowGroupUtil.java
> +added methods generateLongColumnVector and generateDoubleColumnVector for generating the respective column vectors with optional nulls and/or repeating values.
> This addresses bug HIVE-4553.
>     https://issues.apache.org/jira/browse/HIVE-4553
> Diffs
> -----
>   ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/CodeGen.java 53d9a7a
>   ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/TestClass.txt PRE-CREATION
>   ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/TestCodeGen.java PRE-CREATION
>   ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/TestColumnColumnFilterVectorExpressionEvaluation.txt PRE-CREATION
>   ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/TestColumnColumnOperationVectorExpressionEvaluation.txt PRE-CREATION
>   ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/TestColumnScalarFilterVectorExpressionEvaluation.txt PRE-CREATION
>   ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/TestColumnScalarOperationVectorExpressionEvaluation.txt PRE-CREATION