Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Drill, mail # dev - Next steps on the REST API


Copy link to this message
-
Re: Next steps on the REST API
Srihari Srinivasan 2013-08-19, 13:53
Folks,

Any news on this feature? Is there a version of DrillClient I can start
working with?

Hari

On Fri, Aug 9, 2013 at 6:19 PM, Srihari Srinivasan <
[EMAIL PROTECTED]> wrote:

> Jacques,
>
>    - I've created this JIRA for the DrillClient changes<https://issues.apache.org/jira/browse/DRILL-164>.
>    We could discuss the design further on JIRA too.
>    - For the Singleton solution I've created a class called
>    DrillClientProxy that wraps a single DrillClient instance.
>    - As far as the localhost assumption goes what I am taking away is the
>    DrillClient will be able to figure things out itself with the configuration
>    its been given. The http layer need not do anything special.
>
> Hari
>
> On Thu, Aug 8, 2013 at 11:33 PM, Jacques Nadeau <[EMAIL PROTECTED]>wrote:
>
>> > Any thoughts on this?
>>
>> Sorry,  you response was sitting in a draft...
>>
>> >    - Is there any overhead to instantiating a new DrillClient,
>> connecting
>> >    to a drillbit and closing the connection per http request? Jersey
>> creates a
>> >    new instance of the Resource class for each request and the
>> DrillClient
>> >    will get instantiated per request unless we make it a static
>> variable.
>>
>> You probably should avoid this. It will increase latency as the
>> DrillClient is a treadsafe  resource that uses ZooKeeper to find
>> Drillbits.
>>
>> >    - I am assuming that the DrillClient which will be instantiated from
>> the
>> >    http layer will always connect to the co-located/localhost Drillbit
>>
>> I don't think you need to assume this.  It may run locally or on a
>> different node.  DrillClient should figure things out and make sure
>> that everything runs well.
>>
>> >
>> > With respect to the DrillClient we'll need the following methods to
>> support
>> > the async style interaction. If we agree to supporting this design I'll
>> > create a JIRA for this -
>> >
>> >    - new DrillClient().submitQuery(QueryType, Query String) that returns
>> >    the *QueryId* *and continues to execute in the background*
>> >    - new DrillClient().getStatus(queryId) that returns a Status object.
>> It
>> >    could just return *IN_PROGRESS, SUCCEEDED and FAILED* for now. And
>> some
>> >    way to know why it failed in case it did.
>> >    - new DrillClient().getResults(queryId) that returns
>> > *QueryResultBatch*as it is doing now.
>> >
>>
>> I think these make sense.  Parts of this are already stubbed out in
>> the interface.  I don't want to maintain this kind of state on the
>> server but within a particular approach to the Drill client is fine.
>> I think this would be helpful for multiple consumers.  Put together a
>> JIRA and we can work through the details further.
>>
>> One note that we can work out there: I think a paginated or continue
>> interface might be better since the results come back in chunks.
>>
>> thanks,
>> Jacques
>>
>> > While the api I've speced out above is the ideal one I guess this will
>> > require more time and effort to implement it as suggested. Especially
>> for
>> > the getStatus and getResults methods as it implies that the state of a
>> > query's lifecycle will be held/persisted by someone. Is the server
>> already
>> > maintaining some state for tracking the lifecycle of a query which can
>> be
>> > exposed?
>> >
>> > Hari
>>
>
>