Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Drill >> mail # dev >> Re: [jira] [Updated] (DRILL-60) multiple query support


Copy link to this message
-
Re: [jira] [Updated] (DRILL-60) multiple query support
Jacques,

The review board got me. will try tomorrow.

This piece of work is very useful in our case. Thanks in advance for any
comments!

On Wed, Jul 31, 2013 at 1:15 AM, Lisen Mu (JIRA) <[EMAIL PROTECTED]> wrote:

>
>      [
> https://issues.apache.org/jira/browse/DRILL-60?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel]
>
> Lisen Mu updated DRILL-60:
> --------------------------
>
>     Attachment: DRILL-60.0.patch.txt
>
> The review board complains about
>
> 'The file
> 'sandbox/prototype/exec/java-exec/src/main/codegen/ValueVectors/templates/FixedValueVectors.java'
> (r13fa5eb) could not be found in the repository'
>
> I would deal with review board tomorrow. please see the attachment first.
>
> union pop related commits included, for a test case that finally makes
> sense.
>
>
>
> > multiple query support
> > ----------------------
> >
> >                 Key: DRILL-60
> >                 URL: https://issues.apache.org/jira/browse/DRILL-60
> >             Project: Apache Drill
> >          Issue Type: New Feature
> >            Reporter: Lisen Mu
> >         Attachments: DRILL-60.0.patch.txt
> >
> >
> > parse/optimize several SQL(or other QL)s into one logical plan and
> execute them once.
> >
> > * use case
> > If a report generating system is used as a frontend system of drill
> instead of a command line, There would be a set of similar SQL to execute
> to generate a report(consider each day's DAU statistics with geoip
> distribution(groupby) for 7 continuous days). If processed individually,
> plans for SQLs in this kind of set are similar(scan pv log in these days)
> or same(scan GeoIP table) in majority. If processed together, duplicated
> scan(GeoIP) could be eliminated to reduce I/O, or similar scan(pv) could be
> merged to achieve better throughput, and provides chances of other
> intermediate result reuse.
> > * possibility
> > Query in a QL is represented as a tree. Drill models logical plan and
> physical plan as DAG, which is more expressive than tree. More precisely, a
> set of trees(with common nodes or common edges) can be union-ed into a dag.
> So, the current drill logical/physical plan model supports multiple query
> already. Each sql would have a corresponding StoreOP as its result.
> > * problems
> > To do so, physical operators must support multiple iteration over its
> output. That would have side effect on mem management.
>
> --
> This message is automatically generated by JIRA.
> If you think it was sent incorrectly, please contact your JIRA
> administrators
> For more information on JIRA, see: http://www.atlassian.com/software/jira
>