-Re: [jira] [Updated] (DRILL-60) multiple query support
Lisen Mu 2013-07-30, 17:24
The review board got me. will try tomorrow.
This piece of work is very useful in our case. Thanks in advance for any
On Wed, Jul 31, 2013 at 1:15 AM, Lisen Mu (JIRA) <[EMAIL PROTECTED]> wrote:
> Lisen Mu updated DRILL-60:
> Attachment: DRILL-60.0.patch.txt
> The review board complains about
> 'The file
> (r13fa5eb) could not be found in the repository'
> I would deal with review board tomorrow. please see the attachment first.
> union pop related commits included, for a test case that finally makes
> > multiple query support
> > ----------------------
> > Key: DRILL-60
> > URL: https://issues.apache.org/jira/browse/DRILL-60
> > Project: Apache Drill
> > Issue Type: New Feature
> > Reporter: Lisen Mu
> > Attachments: DRILL-60.0.patch.txt
> > parse/optimize several SQL(or other QL)s into one logical plan and
> execute them once.
> > * use case
> > If a report generating system is used as a frontend system of drill
> instead of a command line, There would be a set of similar SQL to execute
> to generate a report(consider each day's DAU statistics with geoip
> distribution(groupby) for 7 continuous days). If processed individually,
> plans for SQLs in this kind of set are similar(scan pv log in these days)
> or same(scan GeoIP table) in majority. If processed together, duplicated
> scan(GeoIP) could be eliminated to reduce I/O, or similar scan(pv) could be
> merged to achieve better throughput, and provides chances of other
> intermediate result reuse.
> > * possibility
> > Query in a QL is represented as a tree. Drill models logical plan and
> physical plan as DAG, which is more expressive than tree. More precisely, a
> set of trees(with common nodes or common edges) can be union-ed into a dag.
> So, the current drill logical/physical plan model supports multiple query
> already. Each sql would have a corresponding StoreOP as its result.
> > * problems
> > To do so, physical operators must support multiple iteration over its
> output. That would have side effect on mem management.
> This message is automatically generated by JIRA.
> If you think it was sent incorrectly, please contact your JIRA
> For more information on JIRA, see: http://www.atlassian.com/software/jira