Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - Enable Snappy compression - not able to load the libs on startup


Copy link to this message
-
Re: Enable Snappy compression - not able to load the libs on startup
Asaf Mesika 2012-07-11, 18:07
start-hbase is wrapper script on of hbase-daemon, which is on top of hbase. HBase itself takes, in some cases (as shown below), environment variables from hadoop shel script.
The first thing you need to check is what I wrote before: the value of the -Djava.library.path using "ps -ef|grep hbase".
--
Asaf Mesika
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
On Tuesday 10 July 2012 at 20:04, Arvid Warnecke wrote:

> Hello Asaf,
>
> On Tue, Jul 10, 2012 at 02:20:03PM +0300, Asaf Mesika wrote:
> > On Jul 10, 2012, at 8:57 AM, Arvid Warnecke wrote:
> > > On Mon, Jul 09, 2012 at 09:10:12PM +0300, Asaf Mesika wrote:
> > > > On Jul 9, 2012, at 21:00 PM, Harsh J wrote:
> > > > > The hbase-daemon.sh (http://hbase-daemon.sh) does not ssh back into the host, so preserves any
> > > > > environment variables you haven't otherwise set in the hbase-env.sh (http://hbase-env.sh)
> > > > > file. I guess that did the trick for you.
> > > > >
> > > >
> > > > Maybe you should look at the content of the jvm argument switch
> > > > -Djava.library.path, (ps -ef | grep hbase , to see the command line).
> > > > This will give you a hint on the directories the .so object is being
> > > > looked for.
> > > >
> > >
> > > It seems that that switch is only in the 'hbase' script itself. But
> > > something like that must be the difference, because in my shell I only
> > > set $HADOOP_HOME and $HBASE_HOME and $HADOOP_CLASSPATH via ~/.zshrc.
> > >
> >
> > It's not only there. Since inside hbase file, you see the following bash section which pulls the value of java.library.path from the hadoop shell script:
> >
> > #If avail, add Hadoop to the CLASSPATH and to the JAVA_LIBRARY_PATH
> > HADOOP_IN_PATH=$(PATH="${HADOOP_HOME:-${HADOOP_PREFIX}}/bin:$PATH" which hadoop 2>/dev/null)
> > if [ -f ${HADOOP_IN_PATH} ]; then
> > HADOOP_JAVA_LIBRARY_PATH=$(HADOOP_CLASSPATH="$CLASSPATH" ${HADOOP_IN_PATH} \
> > org.apache.hadoop.hbase.util.GetJavaProperty java.library.path 2>/dev/null)
> > if [ -n "$HADOOP_JAVA_LIBRARY_PATH" ]; then
> > JAVA_LIBRARY_PATH=$(append_path "${JAVA_LIBRARY_PATH}" "$HADOOP_JAVA_LIBRARY_PATH")
> > fi
> > CLASSPATH=$(append_path "${CLASSPATH}" `${HADOOP_IN_PATH} classpath 2>/dev/null`)
> > fi
> >
> > if [ -d "${HBASE_HOME}/build/native" -o -d "${HBASE_HOME}/lib/native" ]; then
> > if [ -z $JAVA_PLATFORM ]; then
> > JAVA_PLATFORM=`CLASSPATH=${CLASSPATH} ${JAVA} org.apache.hadoop.util.PlatformName | sed -e "s/ /_/g"`
> > fi
> > if [ -d "$HBASE_HOME/build/native" ]; then
> > JAVA_LIBRARY_PATH=$(append_path "$JAVA_LIBRARY_PATH" ${HBASE_HOME}/build/native/${JAVA_PLATFORM}/lib)
> > fi
> >
> > if [ -d "${HBASE_HOME}/lib/native" ]; then
> > JAVA_LIBRARY_PATH=$(append_path "$JAVA_LIBRARY_PATH" ${HBASE_HOME}/lib/native/${JAVA_PLATFORM})
> > fi
> > fi
> >
>
> Thank you. So the start-hbase.sh (http://start-hbase.sh) file should take care of such things,
> too. Or it might be easier to write a wrapper script to call
> hbase-daemon.sh (http://hbase-daemon.sh) for master and regionserver in a row.
>
> Cheers,
> Arvid
>
> --
> [ Arvid Warnecke ][ arvid (at) nostalgix (dot) org ]
> [ IRC/OPN: "madhatter" ][ http://www.nostalgix.org ]
> ---[ ThreePiO was right: Let the Wookiee win. ]---
>
>
>