I believe eventually we want to support data sources that will allow
updates. While our current formats, and development focus is on HDFS
archival storage, we have long term goals to support arbitrary storage
engines like MySQL and HBase. These systems will be able to accept updates,
and with our goal for full SQL 2003 compliance I assume that includes
Again, I assume this would be far out, but as long as the discussion of
options was open I would bring it up as another level of granularity.
On Sun, Jan 12, 2014 at 3:17 PM, Timothy Chen <[EMAIL PROTECTED]> wrote:
> Why do we even need transactions when we perform no updates?
> On Sun, Jan 12, 2014 at 1:03 PM, Jason Altekruse
> <[EMAIL PROTECTED]>wrote:
> > Hi Aman,
> > Thanks for the feedback. I will be defining both global and session local
> > options in my implementation. We will be assuming that session local
> > settings will be tied to the node a client connection is associated
> with, a
> > loss of the connection will lose the session and all settings. Global
> > settings will be added to distributed cache to persist and apply
> > all instances of Drill in the cluster.
> > Another level of granularity that seems to be supported is transaction
> > local settings, at least for SQL server all settings within a transaction
> > are reset to their defaults hen the transaction has finished. I do not
> > the standard well, but I would assume that transactions are included. In
> > widely distributed system like Drill, I'm uncertain how we are going to
> > able to manage locks necessary to keep transactions consistent. Still for
> > full compliance I assume we will need to visit the issue eventually, and
> > might just look at the right place to store those settings when we have a
> > concept of transactions in Drill.
> > - Jason
> > On Fri, Jan 10, 2014 at 5:28 PM, Aman Sinha <[EMAIL PROTECTED]> wrote:
> > > Hi Jason,
> > > so I suppose there are 3 types of option settings that you might be
> > > considering:
> > > 1. Something like 'SET <option_name> TO <value> ' which would be
> > done
> > > before running a query and would be valid for all subsequent queries
> > in
> > > that session.
> > > 2. A configuration file (e.g drill.conf) that would contain the
> > <key,
> > > value> pairs of options and their values and these would be parsed and
> > > applicable for *ALL* sessions
> > > 3. I suppose another requirement is to support EXPLAIN commands
> > > either 'physical' or 'logical' options.
> > >
> > > I would vote for limiting the types of options supported in Drill. In
> > > general, the database vendors have various proprietary set of options.
> > > would think SQL server is too flexible in terms of supporting the
> > > options... maybe we could look at Postgres (or MySQL) instead which
> has a
> > > more standard single token option name and (for the most part) single
> > > token values too.
> > >
> > > As another point of reference, Oracle supports optimizer 'hints' where
> > you
> > > can specify enabling or disabling specific options (example, enable
> > > join, merge join etc.) for a particular subquery within a query.. but I
> > > don't think at this stage we should worry about that.
> > >
> > > Aman
> > >
> > >
> > > On Fri, Jan 10, 2014 at 2:28 PM, Jason Altekruse
> > > <[EMAIL PROTECTED]>wrote:
> > >
> > > > Hello Drill Team,
> > > >
> > > > I have been working the past few days on implementing the SQL options
> > > > statements within the optiq parser and Drill and unfortunately have
> > hit a
> > > > few major decision points.
> > > >
> > > > I was hoping to make the changes to optiq as minimal as possible, and
> > > > design a simple parsing rule that would work for all future Drill
> > > options.
> > > > Right now I have it implemented with a single identifier as an option
> > > name,
> > > > and an identifier or literal as the value (using various types for