Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Drill >> mail # dev >> Setting up a cluster


+
Alexandre BECHE 2013-07-10, 13:22
+
Jacques Nadeau 2013-07-10, 21:21
+
Ted Dunning 2013-07-11, 00:36
Copy link to this message
-
Re: Setting up a cluster
Hi everybody,

I finally got the cluster ready and almost working properly. Now, I hope to
start the data acquisition by the end of the week. As I understood that the
JSON scanner is much more advanced than the HBase one, I will start using
it.

Now I have a few more questions, maybe dedicated to Michael.
As far as I understood, this webUI http://srvgal85.deri.ie/apache-drill/ is
currently working with elasticsearch (does it mean it will be supported in
the future?). My main question is: how far am I from using it with my own
json data stored in HDFS (ie. using the full stack) and what would be the
first step?

Thanks for your help,
Cheers,
Alex

On Thu, Jul 11, 2013 at 2:36 AM, Ted Dunning <[EMAIL PROTECTED]> wrote:

> Even that dependency change is often not necessary.  If your original
> dependency looks like this:
>
>     <dependency>
>       <groupId>org.apache.hadoop</groupId>
>       <artifactId>hadoop-core</artifactId>
>       <version>1.1.0</version>
>       <scope>provided</scope>
>     </dependency>
>
> then your code should work with either Apache Hadoop or MapR.  All you have
> to do is make sure the jars for the distro you have are in the classpath
> correctly.  This does not package the Hadoop jars into your executable
> which can either be a virtue or a vice depending on your requirements.
>
>
>
> On Wed, Jul 10, 2013 at 2:21 PM, Jacques Nadeau <[EMAIL PROTECTED]>
> wrote:
>
> > Ted gave a good long answer.  The short answer is that you'll need to
> > change the dependency on Hadoop to use the MapR distribution which
> > means changing:
> >
> > the following entry in two files files:
> > sandbox/prototype/exec/java-exec/pom.xml and
> > sandbox/prototype/exec/ref/pom.xml
> >
> >       <groupId>org.apache.hadoop</groupId>
> >       <artifactId>hadoop-core</artifactId>
> >       <version>1.1.0</version>
> >
> > to: (note the change in version number)
> >
> >   <groupId>org.apache.hadoop</groupId>
> >   <artifactId>hadoop-core</artifactId>
> >   <version>1.0.3-mapr-2.1.3.1</version>
> >
> > and adding the mapr repository to the list of available repositories:
> >
> >   <repository>
> >       <id>mapr-releases</id>
> >       <url>http://repository.mapr.com/maven/</url>
> >       <snapshots><enabled>false</enabled></snapshots>
> >       <releases><enabled>true</enabled></releases>
> >   </repository>
> >
> >
> > As an Apache initiative, the goal of Drill is to work on all Hadoop
> > distributions.
> >
> > Jacques
> >
> >
> >
> > On Wed, Jul 10, 2013 at 6:22 AM, Alexandre BECHE
> > <[EMAIL PROTECTED]> wrote:
> > > Dear drill dev,
> > >
> > > As discussed yesterday during the Hangout, I am currently setting up a
> > > cluster using the M3 distribution.
> > > I went through the MapR documentation for the installation and I found
> > that
> > > there is no Namenodes but an HDFS compliant API. What are the impact on
> > > DRILL for that? Is DRILL compatible for both system (native HDFS and
> MapR
> > > custom HDFS)?
> > >
> > > Cheers,
> > > Alex
> >
>
+
Michael Hausenblas 2013-07-16, 21:33
+
Alexandre BECHE 2013-07-17, 07:47
+
Alexandre BECHE 2013-07-22, 12:27
+
Ted Dunning 2013-07-10, 21:02