Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Accumulo >> mail # user >> Accumulo and Bigtop Hadoop


+
Roshan Punnoose 2013-05-09, 17:22
+
Christopher 2013-05-09, 17:34
Copy link to this message
-
Re: Accumulo and Bigtop Hadoop
Accumulo 1.4 does not work against hadoop 2 without changing the poms and
recompiling. Accumulo 1.5, not yet released, will support both with
recompilation. One option is to use Bigtop-0.3.x, which uses hadoop-1.0.1
if I remember correctly.
On Thu, May 9, 2013 at 1:22 PM, Roshan Punnoose <[EMAIL PROTECTED]> wrote:

> I have a feeling this has already been asked, but I could not find the
> answer on google.
>
> Bigtop is installing hadoop 2.0.2-alpha into /usr/lib/hadoop,
> /usr/lib/hadoop-hdfs, /usr/lib/hadoop-yarn, etc. However, Accumulo is setup
> to have the "general.classpaths" of accumulo-site to point
> /usr/lib/hadoop/(share)/(hdfs|yarn)/*.jar.
>
> As a quick workaround I had to edit the accumulo-env.sh and
> accumulo-site.sh:
>
> accumulo-env.sh
> ###for bigtop
> export HADOOP_COMMON_DIR="./"
> export HADOOP_COMMON_LIB_JARS_DIR="lib"
> export HADOOP_COMMON_LIB_NATIVE_DIR="lib/native"
> export HDFS_DIR="./"
> export HDFS_LIB_JARS_DIR="lib"
> export YARN_DIR="./"
> export YARN_LIB_JARS_DIR="lib"
> export MAPRED_DIR="./"
> export MAPRED_LIB_JARS_DIR="lib"
>
> export HADOOP_LIBEXEC_DIR=${HADOOP_LIBEXEC_DIR:-"/usr/lib/hadoop/libexec"}
> export HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-"/etc/hadoop/conf"}
> export HADOOP_COMMON_HOME=${HADOOP_COMMON_HOME:-"/usr/lib/hadoop"}
> export HADOOP_HDFS_HOME=${HADOOP_HDFS_HOME:-"/usr/lib/hadoop-hdfs"}
> export
> HADOOP_MAPRED_HOME=${HADOOP_MAPRED_HOME:-"/usr/lib/hadoop-mapreduce"}
> export HADOOP_YARN_HOME=${HADOOP_YARN_HOME:-"/usr/lib/hadoop-yarn"}
> ###for bigtop
>
> accumulo-site.sh (added to the general.classpaths)
>       $HADOOP_MAPRED_HOME/.*.jar,
>       $HADOOP_MAPRED_HOME/$MAPRED_LIB_JARS_DIR/.*.jar,
>       $HADOOP_YARN_HOME/.*.jar,
>       $HADOOP_YARN_HOME/$YARN_LIB_JARS_DIR/.*.jar,
>       $HADOOP_HDFS_HOME/[^.].*.jar,
>       $HADOOP_HDFS_HOME/$HDFS_LIB_JARS_DIR/.*.jar,
>
> This above is working. Has anyone ran into this and found a better
> solution?
>
> Thanks!
>
> Roshan
>
+
Roshan Punnoose 2013-05-09, 17:32
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB