Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Drill >> mail # dev >> Next steps on the REST API


Copy link to this message
-
Re: Next steps on the REST API
> Any thoughts on this?

Sorry,  you response was sitting in a draft...

>    - Is there any overhead to instantiating a new DrillClient, connecting
>    to a drillbit and closing the connection per http request? Jersey creates a
>    new instance of the Resource class for each request and the DrillClient
>    will get instantiated per request unless we make it a static variable.

You probably should avoid this. It will increase latency as the
DrillClient is a treadsafe  resource that uses ZooKeeper to find
Drillbits.

>    - I am assuming that the DrillClient which will be instantiated from the
>    http layer will always connect to the co-located/localhost Drillbit

I don't think you need to assume this.  It may run locally or on a
different node.  DrillClient should figure things out and make sure
that everything runs well.

>
> With respect to the DrillClient we'll need the following methods to support
> the async style interaction. If we agree to supporting this design I'll
> create a JIRA for this -
>
>    - new DrillClient().submitQuery(QueryType, Query String) that returns
>    the *QueryId* *and continues to execute in the background*
>    - new DrillClient().getStatus(queryId) that returns a Status object. It
>    could just return *IN_PROGRESS, SUCCEEDED and FAILED* for now. And some
>    way to know why it failed in case it did.
>    - new DrillClient().getResults(queryId) that returns
> *QueryResultBatch*as it is doing now.
>

I think these make sense.  Parts of this are already stubbed out in
the interface.  I don't want to maintain this kind of state on the
server but within a particular approach to the Drill client is fine.
I think this would be helpful for multiple consumers.  Put together a
JIRA and we can work through the details further.

One note that we can work out there: I think a paginated or continue
interface might be better since the results come back in chunks.

thanks,
Jacques

> While the api I've speced out above is the ideal one I guess this will
> require more time and effort to implement it as suggested. Especially for
> the getStatus and getResults methods as it implies that the state of a
> query's lifecycle will be held/persisted by someone. Is the server already
> maintaining some state for tracking the lifecycle of a query which can be
> exposed?
>
> Hari