Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Drill, mail # dev - Next steps on the REST API


Copy link to this message
-
Re: Next steps on the REST API
Jacques Nadeau 2013-08-20, 02:19
Haven't gotten to it yet.   I suggest you create a test version that does
what you want it to do and code against that so you aren't blocked. How
does that sound?

Jacques
 On Aug 19, 2013 6:54 AM, "Srihari Srinivasan" <[EMAIL PROTECTED]>
wrote:

> Folks,
>
> Any news on this feature? Is there a version of DrillClient I can start
> working with?
>
> Hari
>
> On Fri, Aug 9, 2013 at 6:19 PM, Srihari Srinivasan <
> [EMAIL PROTECTED]> wrote:
>
> > Jacques,
> >
> >    - I've created this JIRA for the DrillClient changes<
> https://issues.apache.org/jira/browse/DRILL-164>.
> >    We could discuss the design further on JIRA too.
> >    - For the Singleton solution I've created a class called
> >    DrillClientProxy that wraps a single DrillClient instance.
> >    - As far as the localhost assumption goes what I am taking away is the
> >    DrillClient will be able to figure things out itself with the
> configuration
> >    its been given. The http layer need not do anything special.
> >
> > Hari
> >
> > On Thu, Aug 8, 2013 at 11:33 PM, Jacques Nadeau <[EMAIL PROTECTED]
> >wrote:
> >
> >> > Any thoughts on this?
> >>
> >> Sorry,  you response was sitting in a draft...
> >>
> >> >    - Is there any overhead to instantiating a new DrillClient,
> >> connecting
> >> >    to a drillbit and closing the connection per http request? Jersey
> >> creates a
> >> >    new instance of the Resource class for each request and the
> >> DrillClient
> >> >    will get instantiated per request unless we make it a static
> >> variable.
> >>
> >> You probably should avoid this. It will increase latency as the
> >> DrillClient is a treadsafe  resource that uses ZooKeeper to find
> >> Drillbits.
> >>
> >> >    - I am assuming that the DrillClient which will be instantiated
> from
> >> the
> >> >    http layer will always connect to the co-located/localhost Drillbit
> >>
> >> I don't think you need to assume this.  It may run locally or on a
> >> different node.  DrillClient should figure things out and make sure
> >> that everything runs well.
> >>
> >> >
> >> > With respect to the DrillClient we'll need the following methods to
> >> support
> >> > the async style interaction. If we agree to supporting this design
> I'll
> >> > create a JIRA for this -
> >> >
> >> >    - new DrillClient().submitQuery(QueryType, Query String) that
> returns
> >> >    the *QueryId* *and continues to execute in the background*
> >> >    - new DrillClient().getStatus(queryId) that returns a Status
> object.
> >> It
> >> >    could just return *IN_PROGRESS, SUCCEEDED and FAILED* for now. And
> >> some
> >> >    way to know why it failed in case it did.
> >> >    - new DrillClient().getResults(queryId) that returns
> >> > *QueryResultBatch*as it is doing now.
> >> >
> >>
> >> I think these make sense.  Parts of this are already stubbed out in
> >> the interface.  I don't want to maintain this kind of state on the
> >> server but within a particular approach to the Drill client is fine.
> >> I think this would be helpful for multiple consumers.  Put together a
> >> JIRA and we can work through the details further.
> >>
> >> One note that we can work out there: I think a paginated or continue
> >> interface might be better since the results come back in chunks.
> >>
> >> thanks,
> >> Jacques
> >>
> >> > While the api I've speced out above is the ideal one I guess this will
> >> > require more time and effort to implement it as suggested. Especially
> >> for
> >> > the getStatus and getResults methods as it implies that the state of a
> >> > query's lifecycle will be held/persisted by someone. Is the server
> >> already
> >> > maintaining some state for tracking the lifecycle of a query which can
> >> be
> >> > exposed?
> >> >
> >> > Hari
> >>
> >
> >
>