Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Drill >> mail # dev >> HBase Query Support


+
Dhruv 2013-10-28, 12:34
+
Aditya 2013-10-28, 20:36
+
James Taylor 2013-10-28, 20:31
+
Aditya 2013-10-28, 22:16
+
James Taylor 2013-10-28, 22:22
+
Dhruv 2013-10-29, 12:16
Copy link to this message
-
Re: HBase Query Support
Yes.  This is very much what I was thinking as well.  If you're in the bay
area, you should come to the Drill meetup on Monday.  We're hoping to try
to gather together interested parties to talk more.  If not, no problem as
we'll report back any thoughts etc here.

thanks,
Jacques
On Tue, Oct 29, 2013 at 5:16 AM, Dhruv <[EMAIL PROTECTED]> wrote:

> Hi Aditya/James,
>
>     As per my understanding Phoenix (having great features while
> storing[e.g. secondary index] + Schema) can be directly used for data
> insertion (Drill does not come in to picture in this flow).
>
>     And for query retrieval;  first level we can use Phoenix end user API
> in local Drillbit to query data.
>
>     DrillUsers -> DrillBit+EmbaddedPhoenixJDBCClient (local mode, single
> instance) -> {HBase cluster..n}
>
>     In the mean time James can draft a design where DrillBit can use more
> lower level API to benefit from Drill's Distributed operators,Joins and
> ValueVector memory model, and using Phoenix's Hbase side coprocessors and
> custom filters (as operator pushdown)
>
>     DrillUsers -> {DrillBit+EmbaddedPhoenixClient (Distributed mode) ..n}
> -> -> {HBase cluster..n}
>
>     Also we should be able to use HBase Table Schema generated by Phoenix.
>
>     Your thoughts?
>
> -Dhruv
>
>
> On 10/29/2013 03:52 AM, James Taylor wrote:
>
>> Sure, I'd be happy to.
>>
>> Thanks,
>> James
>>
>>
>> On Mon, Oct 28, 2013 at 3:16 PM, Aditya <[EMAIL PROTECTED]> wrote:
>>
>>  James,
>>>
>>> Jacques and I discussed this recently and we concur that it would be a
>>> beneficial to build such an integration between the two projects.
>>>
>>> Would you have some time to have a meeting/hangout with Drill team to
>>> talk
>>> this further?
>>>
>>> Regards,
>>> Aditya...
>>>
>>> Regards,
>>> Aditya...
>>>
>>>
>>> On Mon, Oct 28, 2013 at 1:31 PM, James Taylor <[EMAIL PROTECTED]
>>>
>>>> wrote:
>>>> Hi,
>>>> The idea for the Phoenix integration was to come up with the right API
>>>> to
>>>> be able to plug Phoenix into Drill as the HBase query engine/store.
>>>>
>>> Jacque
>>>
>>>> and I got busy on our respective releases, but now that those are both
>>>>
>>> out,
>>>
>>>> I think we should get back to it. If we did it right, then both products
>>>> would be able to continue to evolve independently, but Drill would have
>>>>
>>> top
>>>
>>>> notch query performance for HBase data and Phoenix would have a new
>>>> customer.
>>>>
>>>> Thoughts/ideas?
>>>>
>>>> Thanks,
>>>> James
>>>>
>>>>
>>>> On Mon, Oct 28, 2013 at 5:34 AM, Dhruv <[EMAIL PROTECTED]>
>>>>
>>> wrote:
>>>
>>>> Hi,
>>>>>
>>>>>   I am interested in contributing towards providing HBase Query support
>>>>>
>>>> in
>>>
>>>> DRILL.
>>>>>
>>>>>   A) Can anyone summarize existing plan+Priority for it? (Or we can
>>>>>
>>>> discuss
>>>>
>>>>> in this thread and I will create WIKI page)
>>>>>
>>>>>   B) I see already following JIRA issues, Anybody working on this
>>>>>
>>>> locally?
>>>
>>>>     B-1)    https://issues.apache.org/**jira/browse/DRILL-15<
>>>>>
>>>> https://issues.apache.org/jira/browse/DRILL-15> last updated 13-Aug-13.
>>>>
>>>>>     B-2)    https://issues.apache.org/**jira/browse/DRILL-88<
>>>>>
>>>> https://issues.apache.org/jira/browse/DRILL-88>Unassigned
>>>>
>>>>>     B-3)    https://issues.apache.org/**jira/browse/DRILL-63<
>>>>>
>>>> https://issues.apache.org/jira/browse/DRILL-63>James Taylor seems
>>>> interested to work
>>>>
>>>>>   C) As per my understanding  implimenting
>>>>>
>>>> org.apache.drill.exec.store.**hbase
>>>>
>>>>> package with following for local mode drill bit should suffice for POC?
>>>>>       HBASERecordReader.java
>>>>>        HBASEGroupScan
>>>>>        HBASEScanBatchCreator
>>>>>        HbaseSchemaProvider
>>>>>        HBASEStorageEngine
>>>>>        HBASEStorageEngineConfig
>>>>>        HBASESubScan
>>>>>
>>>>>   D) What part of https://github.com/**forcedotcom/phoenix<
>>>>>
>>>> https://github.com/forcedotcom/phoenix>can we reuse? Any phonix
+
Dhruv 2013-11-05, 13:29
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB