Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Drill >> mail # dev >> Next steps on the REST API


Copy link to this message
-
Re: Next steps on the REST API
I was thinking on similar lines. I was considering wrapping the DrillClient
up in a class that provided async semantics.

On Tue, Aug 20, 2013 at 7:49 AM, Jacques Nadeau <[EMAIL PROTECTED]> wrote:

> Haven't gotten to it yet.   I suggest you create a test version that does
> what you want it to do and code against that so you aren't blocked. How
> does that sound?
>
> Jacques
>  On Aug 19, 2013 6:54 AM, "Srihari Srinivasan" <[EMAIL PROTECTED]>
> wrote:
>
> > Folks,
> >
> > Any news on this feature? Is there a version of DrillClient I can start
> > working with?
> >
> > Hari
> >
> > On Fri, Aug 9, 2013 at 6:19 PM, Srihari Srinivasan <
> > [EMAIL PROTECTED]> wrote:
> >
> > > Jacques,
> > >
> > >    - I've created this JIRA for the DrillClient changes<
> > https://issues.apache.org/jira/browse/DRILL-164>.
> > >    We could discuss the design further on JIRA too.
> > >    - For the Singleton solution I've created a class called
> > >    DrillClientProxy that wraps a single DrillClient instance.
> > >    - As far as the localhost assumption goes what I am taking away is
> the
> > >    DrillClient will be able to figure things out itself with the
> > configuration
> > >    its been given. The http layer need not do anything special.
> > >
> > > Hari
> > >
> > > On Thu, Aug 8, 2013 at 11:33 PM, Jacques Nadeau <[EMAIL PROTECTED]
> > >wrote:
> > >
> > >> > Any thoughts on this?
> > >>
> > >> Sorry,  you response was sitting in a draft...
> > >>
> > >> >    - Is there any overhead to instantiating a new DrillClient,
> > >> connecting
> > >> >    to a drillbit and closing the connection per http request? Jersey
> > >> creates a
> > >> >    new instance of the Resource class for each request and the
> > >> DrillClient
> > >> >    will get instantiated per request unless we make it a static
> > >> variable.
> > >>
> > >> You probably should avoid this. It will increase latency as the
> > >> DrillClient is a treadsafe  resource that uses ZooKeeper to find
> > >> Drillbits.
> > >>
> > >> >    - I am assuming that the DrillClient which will be instantiated
> > from
> > >> the
> > >> >    http layer will always connect to the co-located/localhost
> Drillbit
> > >>
> > >> I don't think you need to assume this.  It may run locally or on a
> > >> different node.  DrillClient should figure things out and make sure
> > >> that everything runs well.
> > >>
> > >> >
> > >> > With respect to the DrillClient we'll need the following methods to
> > >> support
> > >> > the async style interaction. If we agree to supporting this design
> > I'll
> > >> > create a JIRA for this -
> > >> >
> > >> >    - new DrillClient().submitQuery(QueryType, Query String) that
> > returns
> > >> >    the *QueryId* *and continues to execute in the background*
> > >> >    - new DrillClient().getStatus(queryId) that returns a Status
> > object.
> > >> It
> > >> >    could just return *IN_PROGRESS, SUCCEEDED and FAILED* for now.
> And
> > >> some
> > >> >    way to know why it failed in case it did.
> > >> >    - new DrillClient().getResults(queryId) that returns
> > >> > *QueryResultBatch*as it is doing now.
> > >> >
> > >>
> > >> I think these make sense.  Parts of this are already stubbed out in
> > >> the interface.  I don't want to maintain this kind of state on the
> > >> server but within a particular approach to the Drill client is fine.
> > >> I think this would be helpful for multiple consumers.  Put together a
> > >> JIRA and we can work through the details further.
> > >>
> > >> One note that we can work out there: I think a paginated or continue
> > >> interface might be better since the results come back in chunks.
> > >>
> > >> thanks,
> > >> Jacques
> > >>
> > >> > While the api I've speced out above is the ideal one I guess this
> will
> > >> > require more time and effort to implement it as suggested.
> Especially
> > >> for
> > >> > the getStatus and getResults methods as it implies that the state
> of a
> > >> > query's lifecycle will be held/persisted by someone. Is the server
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB