Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Drill, mail # dev - Next steps on the REST API


+
Srihari Srinivasan 2013-08-07, 13:00
+
Jacques Nadeau 2013-08-08, 18:03
+
Jacques Nadeau 2013-08-08, 18:06
+
Srihari Srinivasan 2013-08-09, 12:49
+
Srihari Srinivasan 2013-08-19, 13:53
+
Jacques Nadeau 2013-08-20, 02:19
Copy link to this message
-
Re: Next steps on the REST API
Srihari Srinivasan 2013-08-20, 07:47
I was thinking on similar lines. I was considering wrapping the DrillClient
up in a class that provided async semantics.

On Tue, Aug 20, 2013 at 7:49 AM, Jacques Nadeau <[EMAIL PROTECTED]> wrote:

> Haven't gotten to it yet.   I suggest you create a test version that does
> what you want it to do and code against that so you aren't blocked. How
> does that sound?
>
> Jacques
>  On Aug 19, 2013 6:54 AM, "Srihari Srinivasan" <[EMAIL PROTECTED]>
> wrote:
>
> > Folks,
> >
> > Any news on this feature? Is there a version of DrillClient I can start
> > working with?
> >
> > Hari
> >
> > On Fri, Aug 9, 2013 at 6:19 PM, Srihari Srinivasan <
> > [EMAIL PROTECTED]> wrote:
> >
> > > Jacques,
> > >
> > >    - I've created this JIRA for the DrillClient changes<
> > https://issues.apache.org/jira/browse/DRILL-164>.
> > >    We could discuss the design further on JIRA too.
> > >    - For the Singleton solution I've created a class called
> > >    DrillClientProxy that wraps a single DrillClient instance.
> > >    - As far as the localhost assumption goes what I am taking away is
> the
> > >    DrillClient will be able to figure things out itself with the
> > configuration
> > >    its been given. The http layer need not do anything special.
> > >
> > > Hari
> > >
> > > On Thu, Aug 8, 2013 at 11:33 PM, Jacques Nadeau <[EMAIL PROTECTED]
> > >wrote:
> > >
> > >> > Any thoughts on this?
> > >>
> > >> Sorry,  you response was sitting in a draft...
> > >>
> > >> >    - Is there any overhead to instantiating a new DrillClient,
> > >> connecting
> > >> >    to a drillbit and closing the connection per http request? Jersey
> > >> creates a
> > >> >    new instance of the Resource class for each request and the
> > >> DrillClient
> > >> >    will get instantiated per request unless we make it a static
> > >> variable.
> > >>
> > >> You probably should avoid this. It will increase latency as the
> > >> DrillClient is a treadsafe  resource that uses ZooKeeper to find
> > >> Drillbits.
> > >>
> > >> >    - I am assuming that the DrillClient which will be instantiated
> > from
> > >> the
> > >> >    http layer will always connect to the co-located/localhost
> Drillbit
> > >>
> > >> I don't think you need to assume this.  It may run locally or on a
> > >> different node.  DrillClient should figure things out and make sure
> > >> that everything runs well.
> > >>
> > >> >
> > >> > With respect to the DrillClient we'll need the following methods to
> > >> support
> > >> > the async style interaction. If we agree to supporting this design
> > I'll
> > >> > create a JIRA for this -
> > >> >
> > >> >    - new DrillClient().submitQuery(QueryType, Query String) that
> > returns
> > >> >    the *QueryId* *and continues to execute in the background*
> > >> >    - new DrillClient().getStatus(queryId) that returns a Status
> > object.
> > >> It
> > >> >    could just return *IN_PROGRESS, SUCCEEDED and FAILED* for now.
> And
> > >> some
> > >> >    way to know why it failed in case it did.
> > >> >    - new DrillClient().getResults(queryId) that returns
> > >> > *QueryResultBatch*as it is doing now.
> > >> >
> > >>
> > >> I think these make sense.  Parts of this are already stubbed out in
> > >> the interface.  I don't want to maintain this kind of state on the
> > >> server but within a particular approach to the Drill client is fine.
> > >> I think this would be helpful for multiple consumers.  Put together a
> > >> JIRA and we can work through the details further.
> > >>
> > >> One note that we can work out there: I think a paginated or continue
> > >> interface might be better since the results come back in chunks.
> > >>
> > >> thanks,
> > >> Jacques
> > >>
> > >> > While the api I've speced out above is the ideal one I guess this
> will
> > >> > require more time and effort to implement it as suggested.
> Especially
> > >> for
> > >> > the getStatus and getResults methods as it implies that the state
> of a
> > >> > query's lifecycle will be held/persisted by someone. Is the server
+
Jacques Nadeau 2013-08-23, 01:00
+
Srihari Srinivasan 2013-09-02, 08:34