Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Accumulo >> mail # user >> Accumulo and Bigtop Hadoop


Copy link to this message
-
Accumulo and Bigtop Hadoop
I have a feeling this has already been asked, but I could not find the
answer on google.

Bigtop is installing hadoop 2.0.2-alpha into /usr/lib/hadoop,
/usr/lib/hadoop-hdfs, /usr/lib/hadoop-yarn, etc. However, Accumulo is setup
to have the "general.classpaths" of accumulo-site to point
/usr/lib/hadoop/(share)/(hdfs|yarn)/*.jar.

As a quick workaround I had to edit the accumulo-env.sh and
accumulo-site.sh:

accumulo-env.sh
###for bigtop
export HADOOP_COMMON_DIR="./"
export HADOOP_COMMON_LIB_JARS_DIR="lib"
export HADOOP_COMMON_LIB_NATIVE_DIR="lib/native"
export HDFS_DIR="./"
export HDFS_LIB_JARS_DIR="lib"
export YARN_DIR="./"
export YARN_LIB_JARS_DIR="lib"
export MAPRED_DIR="./"
export MAPRED_LIB_JARS_DIR="lib"

export HADOOP_LIBEXEC_DIR=${HADOOP_LIBEXEC_DIR:-"/usr/lib/hadoop/libexec"}
export HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-"/etc/hadoop/conf"}
export HADOOP_COMMON_HOME=${HADOOP_COMMON_HOME:-"/usr/lib/hadoop"}
export HADOOP_HDFS_HOME=${HADOOP_HDFS_HOME:-"/usr/lib/hadoop-hdfs"}
export HADOOP_MAPRED_HOME=${HADOOP_MAPRED_HOME:-"/usr/lib/hadoop-mapreduce"}
export HADOOP_YARN_HOME=${HADOOP_YARN_HOME:-"/usr/lib/hadoop-yarn"}
###for bigtop

accumulo-site.sh (added to the general.classpaths)
      $HADOOP_MAPRED_HOME/.*.jar,
      $HADOOP_MAPRED_HOME/$MAPRED_LIB_JARS_DIR/.*.jar,
      $HADOOP_YARN_HOME/.*.jar,
      $HADOOP_YARN_HOME/$YARN_LIB_JARS_DIR/.*.jar,
      $HADOOP_HDFS_HOME/[^.].*.jar,
      $HADOOP_HDFS_HOME/$HDFS_LIB_JARS_DIR/.*.jar,

This above is working. Has anyone ran into this and found a better solution?

Thanks!

Roshan