Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Drill >> mail # dev >> Next steps on the REST API


Copy link to this message
-
Re: Next steps on the REST API
> You probably should avoid this. It will increase latency as the
> DrillClient is a treadsafe  resource that uses ZooKeeper to find
> Drillbits.

The DrillClient is not only treadsafe, it is also threadsafe.  My
suggestion would be to maintain a singleton and inject the reference
to it as particular requests come in.  The start up and shut it down
could be managed through servlet lifecycle or similar.  We're using
Jackson's @JacksonInject for this other places and I'd suggest you
enable Jackson within your http engine and use the same methods there
to minimize different JSON parsers and injection approaches.

J

On Thu, Aug 8, 2013 at 11:03 AM, Jacques Nadeau <[EMAIL PROTECTED]> wrote:
>> Any thoughts on this?
>
> Sorry,  you response was sitting in a draft...
>
>>    - Is there any overhead to instantiating a new DrillClient, connecting
>>    to a drillbit and closing the connection per http request? Jersey creates a
>>    new instance of the Resource class for each request and the DrillClient
>>    will get instantiated per request unless we make it a static variable.
>

>
>>    - I am assuming that the DrillClient which will be instantiated from the
>>    http layer will always connect to the co-located/localhost Drillbit
>
> I don't think you need to assume this.  It may run locally or on a
> different node.  DrillClient should figure things out and make sure
> that everything runs well.
>
>>
>> With respect to the DrillClient we'll need the following methods to support
>> the async style interaction. If we agree to supporting this design I'll
>> create a JIRA for this -
>>
>>    - new DrillClient().submitQuery(QueryType, Query String) that returns
>>    the *QueryId* *and continues to execute in the background*
>>    - new DrillClient().getStatus(queryId) that returns a Status object. It
>>    could just return *IN_PROGRESS, SUCCEEDED and FAILED* for now. And some
>>    way to know why it failed in case it did.
>>    - new DrillClient().getResults(queryId) that returns
>> *QueryResultBatch*as it is doing now.
>>
>
> I think these make sense.  Parts of this are already stubbed out in
> the interface.  I don't want to maintain this kind of state on the
> server but within a particular approach to the Drill client is fine.
> I think this would be helpful for multiple consumers.  Put together a
> JIRA and we can work through the details further.
>
> One note that we can work out there: I think a paginated or continue
> interface might be better since the results come back in chunks.
>
> thanks,
> Jacques
>
>> While the api I've speced out above is the ideal one I guess this will
>> require more time and effort to implement it as suggested. Especially for
>> the getStatus and getResults methods as it implies that the state of a
>> query's lifecycle will be held/persisted by someone. Is the server already
>> maintaining some state for tracking the lifecycle of a query which can be
>> exposed?
>>
>> Hari