Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Drill >> mail # user >> complie the drill with hadoop-2.2


Copy link to this message
-
complie the drill with hadoop-2.2
Hi
  I have complie drill with hadoop-2.2. Here are my steps.

  1. add a profile section to pom.xml:
  <profile>
      <id>apache</id>
      <properties>
        <alt-hadoop>apache</alt-hadoop>
      </properties>
        <dependencies>
          <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-common</artifactId>
            <version>2.2.0</version>
            <exclusions>
              <exclusion>
                <artifactId>commons-logging</artifactId>
                <groupId>commons-logging</groupId>
              </exclusion>
            </exclusions>
          </dependency>
          <dependency>
            <groupId>jline</groupId>
              <artifactId>jline</artifactId>
              <version>2.10</version>
             </dependency>
        </dependencies>
    </profile>

2: complie the src:
    mvn clean install -DskipTests  -Papache
3. after  compile successfully,
   $ cd distribution/
   $ ls
      pom.xml  src  target
   $ cd target/
   $ tar -xf apache-drill-1.0.0-m1-incubating-binary-release.tar.gz
   $ ls apache-drill-1.0.0-m1-incubating/lib/protobuf-java-2.4.1.jar
   apache-drill-1.0.0-m1-incubating/lib/protobuf-java-2.4.1.jar

  You need to  change the  *protobuf-java-2.4.1.jar*  to
*protobuf-java-2.5.0.jar*.
4. add a dfs storage *in conf/storage-engines.json*
   "parquet-dfs" :
      {
        "type":"parquet",
        "dfsName" : "hdfs://hadoop2:8020/drill"
      }
 5.You also need to make some changes in the  *bin/drill-config.sh*

   if [ "${HADOOP_HOME}x" != "x" ]
then
  HADOOP_CLASSPATH=""
  for jar in `ls $HADOOP_HOME/share/hadoop/*/*.jar`
  do
    echo $jar | grep -v -f $DRILL_HOME/bin/hadoop-excludes.txt >/dev/null
    if [ "$?" -eq "0" ]
    then
      HADOOP_CLASSPATH=$jar:$HADOOP_CLASSPATH
    fi
  done
  export HADOOP_CLASSPATH=$HADOOP_HOME/conf:$HADOOP_CLASSPATH
fi

6.$ ./bin/sqlline -u jdbc:drill:schema=parquet-dfs

Loaded singnal handler: SunSignalHandler
/home/drill/.sqlline/sqlline.properties (No such file or directory)
scan complete in 25ms
scan complete in 4053ms
Connecting to jdbc:drill:schema=parquet-dfs
Connected to: Drill (version 1.0)
Driver: Apache Drill JDBC Driver (version 1.0)
Autocommit status: true
Transaction isolation: TRANSACTION_REPEATABLE_READ
sqlline version ??? by Marc Prud'hommeaux
0: jdbc:drill:schema=parquet-dfs> select * from "/drill/region.parquet"
. . . . . . . . . . . . . . . . > ;
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further
details.
+------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
_MAP
|
+------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| {"R_REGIONKEY":0,"R_NAME":"AFRICA","R_COMMENT":"lar deposits. blithely
final packages cajole. regular waters are final requests. regular accounts
are according  |
| {"R_REGIONKEY":1,"R_NAME":"AMERICA","R_COMMENT":"hs use ironic, even
requests.
s"}
|
| {"R_REGIONKEY":2,"R_NAME":"ASIA","R_COMMENT":"ges. thinly even pinto
beans
ca"}
|
| {"R_REGIONKEY":3,"R_NAME":"EUROPE","R_COMMENT":"ly final courts cajole
furiously final
excuse"}                                                                  |
| {"R_REGIONKEY":4,"R_NAME":"MIDDLE EAST","R_COMMENT":"uickly special
accounts cajole carefully blithely close requests. carefully final
asymptotes haggle furious |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------+
5 rows selected (4.928 seconds)
0: jdbc:drill:schema=parquet-dfs>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB