Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Accumulo >> mail # user >> Accumulo and Bigtop Hadoop


Copy link to this message
-
Accumulo and Bigtop Hadoop
I have a feeling this has already been asked, but I could not find the
answer on google.

Bigtop is installing hadoop 2.0.2-alpha into /usr/lib/hadoop,
/usr/lib/hadoop-hdfs, /usr/lib/hadoop-yarn, etc. However, Accumulo is setup
to have the "general.classpaths" of accumulo-site to point
/usr/lib/hadoop/(share)/(hdfs|yarn)/*.jar.

As a quick workaround I had to edit the accumulo-env.sh and
accumulo-site.sh:

accumulo-env.sh
###for bigtop
export HADOOP_COMMON_DIR="./"
export HADOOP_COMMON_LIB_JARS_DIR="lib"
export HADOOP_COMMON_LIB_NATIVE_DIR="lib/native"
export HDFS_DIR="./"
export HDFS_LIB_JARS_DIR="lib"
export YARN_DIR="./"
export YARN_LIB_JARS_DIR="lib"
export MAPRED_DIR="./"
export MAPRED_LIB_JARS_DIR="lib"

export HADOOP_LIBEXEC_DIR=${HADOOP_LIBEXEC_DIR:-"/usr/lib/hadoop/libexec"}
export HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-"/etc/hadoop/conf"}
export HADOOP_COMMON_HOME=${HADOOP_COMMON_HOME:-"/usr/lib/hadoop"}
export HADOOP_HDFS_HOME=${HADOOP_HDFS_HOME:-"/usr/lib/hadoop-hdfs"}
export HADOOP_MAPRED_HOME=${HADOOP_MAPRED_HOME:-"/usr/lib/hadoop-mapreduce"}
export HADOOP_YARN_HOME=${HADOOP_YARN_HOME:-"/usr/lib/hadoop-yarn"}
###for bigtop

accumulo-site.sh (added to the general.classpaths)
      $HADOOP_MAPRED_HOME/.*.jar,
      $HADOOP_MAPRED_HOME/$MAPRED_LIB_JARS_DIR/.*.jar,
      $HADOOP_YARN_HOME/.*.jar,
      $HADOOP_YARN_HOME/$YARN_LIB_JARS_DIR/.*.jar,
      $HADOOP_HDFS_HOME/[^.].*.jar,
      $HADOOP_HDFS_HOME/$HDFS_LIB_JARS_DIR/.*.jar,

This above is working. Has anyone ran into this and found a better solution?

Thanks!

Roshan
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB