Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Drill >> mail # dev >> Building logical plans

Copy link to this message
Re: Building logical plans
On Feb 14, 2013, at 8:49 AM, Jacques Nadeau <[EMAIL PROTECTED]> wrote:

> Julian, you're definitely the first to do it.  The only place where I know
> things are a little awkward is the storage engine to operator stichting
> occurs.  The fact that everything is final is also a bit challenging
> depending on how you're building things.  I'd like to make it cleaner to
> ease your pain.  Where else were you having problems?

The main problems are in getting the operator graph to validate. Initially I made each node a parent of its predecessor, then I noticed that the examples use a sequence. (Seems like a redundant concept?) I tried to build a sequence, then saw that org.apache.drill.logical.data.Sequence is abstract.

Long term I'd prefer to use the classes (or a builder based on them) because I like type-safety. Short term I think I'm going to generate JSON strings by hand, as Tim suggested. You can take a look at the code I write and see whether it can be converted into a builder.

I think builder is probably necessary because certain fields such as ids and references are final, whereas they are dynamic while the plan is being constructed. There are data structures in the plan that are designed for plan execution, so its design seems skewed towards reading rather than writing JSON. That's reasonable; just saying.

> Also, I need to push some updates I've done so that the syntax of the plan
> matches that of the spec.

OK, I'll watch for your push.