Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Drill >> mail # dev >> Setting up a cluster


Copy link to this message
-
Re: Setting up a cluster

> As far as I understood, this webUI http://srvgal85.deri.ie/apache-drill/ is
> currently working with elasticsearch (does it mean it will be supported in
> the future?).
Note that the above URL was just a demo deployment. As soon as DRILL-77 [1], the REST API, is resolved (or close to) I'll port the UI over to it. ES was just necessary back then (Oct last year IIRC) as no backend existed.

You can track the progress re the WebUI by subscribing to DRILL-58 [2]. I would think that Hari soon has a breakthrough with the REST API and this means I should get around to this also rather soonish.

Hari, any thoughts re timing?
Cheers,
Michael

[1] https://issues.apache.org/jira/browse/DRILL-77
[2] https://issues.apache.org/jira/browse/DRILL-58

--
Michael Hausenblas
Ireland, Europe
http://mhausenblas.info/

On 16 Jul 2013, at 22:26, Alexandre BECHE <[EMAIL PROTECTED]> wrote:

> Hi everybody,
>
> I finally got the cluster ready and almost working properly. Now, I hope to
> start the data acquisition by the end of the week. As I understood that the
> JSON scanner is much more advanced than the HBase one, I will start using
> it.
>
> Now I have a few more questions, maybe dedicated to Michael.
> As far as I understood, this webUI http://srvgal85.deri.ie/apache-drill/ is
> currently working with elasticsearch (does it mean it will be supported in
> the future?). My main question is: how far am I from using it with my own
> json data stored in HDFS (ie. using the full stack) and what would be the
> first step?
>
> Thanks for your help,
> Cheers,
> Alex
>
>
>
> On Thu, Jul 11, 2013 at 2:36 AM, Ted Dunning <[EMAIL PROTECTED]> wrote:
>
>> Even that dependency change is often not necessary.  If your original
>> dependency looks like this:
>>
>>    <dependency>
>>      <groupId>org.apache.hadoop</groupId>
>>      <artifactId>hadoop-core</artifactId>
>>      <version>1.1.0</version>
>>      <scope>provided</scope>
>>    </dependency>
>>
>> then your code should work with either Apache Hadoop or MapR.  All you have
>> to do is make sure the jars for the distro you have are in the classpath
>> correctly.  This does not package the Hadoop jars into your executable
>> which can either be a virtue or a vice depending on your requirements.
>>
>>
>>
>> On Wed, Jul 10, 2013 at 2:21 PM, Jacques Nadeau <[EMAIL PROTECTED]>
>> wrote:
>>
>>> Ted gave a good long answer.  The short answer is that you'll need to
>>> change the dependency on Hadoop to use the MapR distribution which
>>> means changing:
>>>
>>> the following entry in two files files:
>>> sandbox/prototype/exec/java-exec/pom.xml and
>>> sandbox/prototype/exec/ref/pom.xml
>>>
>>>      <groupId>org.apache.hadoop</groupId>
>>>      <artifactId>hadoop-core</artifactId>
>>>      <version>1.1.0</version>
>>>
>>> to: (note the change in version number)
>>>
>>>  <groupId>org.apache.hadoop</groupId>
>>>  <artifactId>hadoop-core</artifactId>
>>>  <version>1.0.3-mapr-2.1.3.1</version>
>>>
>>> and adding the mapr repository to the list of available repositories:
>>>
>>>  <repository>
>>>      <id>mapr-releases</id>
>>>      <url>http://repository.mapr.com/maven/</url>
>>>      <snapshots><enabled>false</enabled></snapshots>
>>>      <releases><enabled>true</enabled></releases>
>>>  </repository>
>>>
>>>
>>> As an Apache initiative, the goal of Drill is to work on all Hadoop
>>> distributions.
>>>
>>> Jacques
>>>
>>>
>>>
>>> On Wed, Jul 10, 2013 at 6:22 AM, Alexandre BECHE
>>> <[EMAIL PROTECTED]> wrote:
>>>> Dear drill dev,
>>>>
>>>> As discussed yesterday during the Hangout, I am currently setting up a
>>>> cluster using the M3 distribution.
>>>> I went through the MapR documentation for the installation and I found
>>> that
>>>> there is no Namenodes but an HDFS compliant API. What are the impact on
>>>> DRILL for that? Is DRILL compatible for both system (native HDFS and
>> MapR
>>>> custom HDFS)?
>>>>
>>>> Cheers,
>>>> Alex
>>>
>>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB