Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Drill, mail # dev - Review Request 14958: Patch for DRILL-229


+
Ben Becker 2013-10-25, 22:58
Copy link to this message
-
Re: Review Request 14958: Patch for DRILL-229
Steven Phillips 2013-10-25, 23:28

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14958/#review27561
-----------------------------------------------------------

exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/MergingReceiverPOP.java
<https://reviews.apache.org/r/14958/#comment53499>

    Should we use OrderDef, rather than LogicalExpression here? The incoming batches will be sorted based on OrderDef, which is just a LogicalExpression with an Order. As is, this won't be able to handle the case where the data was sorted DESC, will it?
- Steven Phillips
On Oct. 25, 2013, 10:58 p.m., Ben Becker wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14958/
> -----------------------------------------------------------
>
> (Updated Oct. 25, 2013, 10:58 p.m.)
>
>
> Review request for drill.
>
>
> Bugs: DRILL-229
>     https://issues.apache.org/jira/browse/DRILL-229
>
>
> Repository: drill-git
>
>
> Description
> -------
>
> Implements the merging receiver operator as described in DRILL-229.
>
> Commit Log:
>
>  - Fix multi-batch bugs.  Add human-readable test output.
>  - Fix recordCount and valueIndex off-by-one
>  - Implement manual comparison code generation.  Manual approach was chosen over modifying the ValueVectorReadExpression, EvalutionVisitor and CodeGenerator to minimize special-case code in those classes.
>  - Implemented code gen for copying; comparison needs a materializer that works on a RecordBatchLoader
>  - Flesh out the reset of the merging receiver and fix bugs.  Next step is generated code for comparator and copier.
>  - Implements support for RecordBatchLoader, VectorContainer and BatchSchema in MergingRecordBatch Implements first-iteration specific logic WIP: Merge logic.  Some constructs are in place, others are temporary or only stubbed out)
>  - Add SingleMergeExchange, which constructs one receiver and many senders (similar to UnionExchange)
>  - Implement stub for MergingRecordBatch
>  - Add test for MergingReceiver (leveraging SingleMergeExchange) s/MergingReceiver/MergingReceiverPOP/
>  - Implement basic merging POP config and creator
>
>
> Diffs
> -----
>
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/AbstractPhysicalVisitor.java 286144b
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/PhysicalVisitor.java a36b65a
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/MergingReceiverPOP.java PRE-CREATION
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/SingleMergeExchange.java PRE-CREATION
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/ImplCreator.java 94acc0e
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/MergingReceiverCreator.java PRE-CREATION
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/mergereceiver/MergingReceiverGeneratorBase.java PRE-CREATION
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/mergereceiver/MergingReceiverTemplate.java PRE-CREATION
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/mergereceiver/MergingRecordBatch.java PRE-CREATION
>   exec/java-exec/src/main/java/org/apache/drill/exec/record/VectorContainer.java 939245b
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/mergereceiver/TestMergingReceiver.java PRE-CREATION
>   exec/java-exec/src/test/resources/mergerecv/merging_receiver.json PRE-CREATION
>
> Diff: https://reviews.apache.org/r/14958/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Ben Becker
>
>

+
Ben Becker 2013-10-30, 00:06
+
Jacques Nadeau 2013-11-10, 01:21
+
Ben Becker 2013-11-11, 17:11
+
Ben Becker 2013-11-11, 17:11
+
Ben Becker 2013-11-12, 00:31