Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Drill >> mail # dev >> Review Request 15564: Patch for DRILL-274


Copy link to this message
-
Re: Review Request 15564: Patch for DRILL-274
Sorry got no internet access on my laptop so have to put comments here in email:

1, from the rawspoolbuffer class I see it's intended for multithreaded with volatiles, but I have question when multiple threads get from the buffer. Looks like when the buffer is empty and is unfinished, it blocks until it finds an item or a interrupt exception happens. I wonder if it is marked finished while it's blocked? Does it hang forever since there is no timeout on the take?

2, the spooling test doesn't seem to test spooling as I think it will pass even without spooling given enough direct memory right? Perhaps should verify the spooled data is same as expected?

Tim

Sent from my iPhone

> On Nov 14, 2013, at 6:00 PM, "Steven Phillips" <[EMAIL PROTECTED]> wrote:
>
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/15564/
> -----------------------------------------------------------
>
> Review request for drill.
>
>
> Bugs: DRILL-274
>    https://issues.apache.org/jira/browse/DRILL-274
>
>
> Repository: drill-git
>
>
> Description
> -------
>
> DRILL-274: Spooling batch buffer
>
>
> Diffs
> -----
>
>  distribution/src/resources/drill-override.conf 7694cedf62ae5dde1221d3bef0ac6cdaedaf78d4
>  exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java 36504f6df36d3c3bf7bf6b158f777777125fef74
>  exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java a7f6d2ef03eb2fef20fed5f327a12ea3c82989de
>  exec/java-exec/src/main/java/org/apache/drill/exec/store/LocalSyncableFileSystem.java PRE-CREATION
>  exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/AbstractFragmentCollector.java d58de2fc1b7a3369cd904433ecd87f3c7d1019ec
>  exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/BitComHandlerImpl.java 5639851f4c99406f210b1d65c1c8ae3739f9dcfc
>  exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/IncomingBuffers.java c9e56085aeebfeb44b2ae82c981251a7d3e47675
>  exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/MergingCollector.java 670347cba07b4c5010053a89a6352973d40a5dca
>  exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/PartitionedCollector.java af127781bb1d6bfab312d2843a3d41eadbf9f4f4
>  exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/SpoolingRawBatchBuffer.java PRE-CREATION
>  exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/UnlimitedRawBatchBuffer.java 43870da545a4659f85bf608c4d2fda1f808765cf
>  exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/RunningFragmentManager.java 588316b3e5e3f0d9f35d6fb30070001ddb539f4e
>  exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/RemoteFragmentHandler.java d947d683d4686cabf30b85b927ed8fe3847ef6f1
>  exec/java-exec/src/main/resources/drill-module.conf 725c6b4d41bb0b24876e74e7aed7a83f5e6f42f5
>  exec/java-exec/src/test/java/org/apache/drill/exec/fn/impl/TestRepeatedFunction.java ac05319725e0bf797a67a133f44a9a6988ab12c2
>  exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestComparisonFunctions.java c47fa89ab65b18ca21fd4b2f2dc9d478b5fe4809
>  exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestOptiqPlans.java 7e7ac633aacd70469cbf83f4b52641853d690f1f
>  exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestSimpleFunctions.java cbb3e5495fdf0a3efc093646c461b9216e2bc486
>  exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/agg/TestAgg.java 7ce69e2bf7248aa78444d89524c3d4d06e275835
>  exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/filter/TestSimpleFilter.java e81774ad74f2eeeeb78e68216c5ab0a3b60f6219
>  exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/join/TestMergeJoin.java ceebe559f5f9d605aab13a1a204443249c012292
>  exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/limit/TestSimpleLimit.java ef7638ede343b055a6686ffaa7f7d6d72a97c118