Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Drill >> mail # dev >> Hangout Notes, and a question


Copy link to this message
-
Re: Hangout Notes, and a question
The logical and physical setup makes sense, but I guess the only thing I'm
still unsure about is how we transfer the setting data from the optiq SQL
parse tree into Drill. Currently the SQL tree is transformed directly into
a logical plan, I'm just uncertain if there is another path out of optiq
for passing the options that we are interested in using.

I had seen the plan fragments formatted like physical plans, so I assumed
they were still passed as JSON, I had not realized we had defined protobuf
structs to hold them.

Thanks,
- Jason
On Wed, Jan 22, 2014 at 11:59 AM, Jacques Nadeau <[EMAIL PROTECTED]> wrote:

> It seems that you're somewhat mixing internal formats and external apis.
>  You can submit a query via sql, logical or physical.  For sql, you set
> options via separate statements.  For logical and physical you set options
> via the head of the plan.  Drill should internally manage the options
> information to pass to other nodes as part of query execution.  In that
> case, the data will be a protobuf struct as part of PlanFragment as opposed
> to anything to do with logical or physical plans.  In the context of
> PlanFragment, options are only relevant to that fragment and no 'session'
> state is maintained on those working nodes.
>
>
>
>
> On Wed, Jan 22, 2014 at 9:38 AM, Jason Altekruse
> <[EMAIL PROTECTED]>wrote:
>
> > I remember having discussed this. I was thinking this was related to
> > distributing the settings to other drillbits that would need to use
> > settings in their various plan fragments, but I that would work for
> keeping
> > all of the options consistent.
> >
> > In that case if set statements in SQL will be independent statements,
> will
> > we be passing around plans with no bodies to set the options originally?
> >
> > Thanks,
> > Jason
> >
> >
> > On Wed, Jan 22, 2014 at 11:34 AM, Jacques Nadeau <[EMAIL PROTECTED]>
> > wrote:
> >
> > > My thoughts were as part of the plan but not as part of the operator.
> >  Put
> > > it in the head as an additional set of data.
> > >
> > >
> > > On Wed, Jan 22, 2014 at 9:30 AM, Jason Altekruse
> > > <[EMAIL PROTECTED]>wrote:
> > >
> > > > Hi everyone,
> > > >
> > > > I have updated the google doc with notes from this weeks meeting.
> > > >
> > > >
> > > >
> > >
> >
> https://docs.google.com/document/d/1o2GvZUtJvKzN013JdM715ZBzhseT0VyZ9WgmLMeeUUk
> > > >
> > > > In response to my question about how we want to implement setting
> > options
> > > > within Drill we had a rather long discussion about storing
> distributed
> > > > state in Drill.
> > > >
> > > > Unfortunately during the beginning of the discussion when I was
> > > > participating more actively I wasn't taking complete notes. I kind of
> > > lost
> > > > the answer to my original question with all of the other topics we
> > > > discussed.
> > > >
> > > > Did we want to implement setting options as a component of both the
> > > logical
> > > > and physical plans, or separated from the usual plan processing path?
> > > >
> > > > -Jason Altekruse
> > > >
> > >
> >
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB