Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Drill, mail # dev - Review Request 15111: Patch for DRILL-271


Copy link to this message
-
Re: Review Request 15111: Patch for DRILL-271
Steven Phillips 2013-11-01, 00:57


> On Oct. 31, 2013, 7:12 a.m., Mehant Baid wrote:
> > exec/java-exec/src/main/java/org/apache/drill/exec/cache/VectorAccessibleSerializable.java, line 178
> > <https://reviews.apache.org/r/15111/diff/1/?file=374556#file374556line178>
> >
> >     I am not sure if its a good idea to combine ByteBuf memory management and its serialization into one class. Seems to me it'd be the responsibility of the client of this class to release or retain the buffers.

I reworked things, so now VectorAccessibleSerializable take a WritableBatch object. The releasing a buffers and reconstruction of vectors is done in the WritableBatch class.
- Steven
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15111/#review27859
-----------------------------------------------------------
On Nov. 1, 2013, 12:35 a.m., Steven Phillips wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/15111/
> -----------------------------------------------------------
>
> (Updated Nov. 1, 2013, 12:35 a.m.)
>
>
> Review request for drill.
>
>
> Bugs: DRILL-271
>     https://issues.apache.org/jira/browse/DRILL-271
>
>
> Repository: drill-git
>
>
> Description
> -------
>
> DRILL-271 Address code review comments. VectorAccessibleSerializable now take WritableBatch. Release and reconstruct code now part of Writablebatch class.
>
>
> DRILL-271 abstract out serialization in trace record batch by using VectorAccessibleSerializable. Add ability to retain vectors.
>
>
> DRILL-271 release buffer in VectorAccessibleSerializable read. Also read directly from stream.
>
>
> DRILL-271 refactor DistributedCache code. Uses hazel cast 3.1 and custom serialization.
>
>
> DRILL-271 retool VectorContainerSerializable to work with containers and batches. also modify DrillSerializable to work with InputStream, OutputStream
>
>
> Addressed review comments from Jacques
>
>
> DRILL-256 revised patch
>
>
> Diffs
> -----
>
>   distribution/src/resources/drill-override.conf c2ed9df26b63ad81b06aa665af765e67ec930656
>   exec/java-exec/pom.xml 063e60e038beb2c3100848985afea9c651af9ca6
>   exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java 72776d1a79cdb58241a9de72cb985462b3579068
>   exec/java-exec/src/main/java/org/apache/drill/exec/cache/DrillSerializable.java 534d78106fd61ab16a428e6d46bced563724fb15
>   exec/java-exec/src/main/java/org/apache/drill/exec/cache/HCDrillSerializableWrapper.java 3f2c41c5e61fd097a697b6c31a3de20b307d1b50
>   exec/java-exec/src/main/java/org/apache/drill/exec/cache/HCSerializableWrapperClasses.java d22723ab021259197c8fa4f2cc23270301990de4
>   exec/java-exec/src/main/java/org/apache/drill/exec/cache/HCVectorAccessibleSerializer.java PRE-CREATION
>   exec/java-exec/src/main/java/org/apache/drill/exec/cache/HazelCache.java 577dfebb1e9108650ea72b00f844748de3de56fb
>   exec/java-exec/src/main/java/org/apache/drill/exec/cache/LocalCache.java 7ad6ec687e02a90e4936bcae16e8ef666818c327
>   exec/java-exec/src/main/java/org/apache/drill/exec/cache/ProtoBufWrap.java 4aea645afd278a6b0233eff19487dbd66c6a0272
>   exec/java-exec/src/main/java/org/apache/drill/exec/cache/VectorAccessibleSerializable.java PRE-CREATION
>   exec/java-exec/src/main/java/org/apache/drill/exec/cache/VectorContainerSerializable.java 1e6eeacec76ebd434720849b669735c9cf82df05
>   exec/java-exec/src/main/java/org/apache/drill/exec/client/QuerySubmitter.java 160ef7ff21b079cd0e51aa5b62af09c96e932b5f
>   exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java 352c467a8f26183631f03a89c5cf21c1df18b3ac
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/AbstractPhysicalVisitor.java bf6c68ca2d67d8d40001ee21656345ceb823ae75
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/PhysicalVisitor.java 5692b9f0827eec39671a29cee064e8247b7d344a