Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - Enable Snappy compression - not able to load the libs on startup


Copy link to this message
-
Re: Enable Snappy compression - not able to load the libs on startup
Asaf Mesika 2012-07-10, 11:20

On Jul 10, 2012, at 8:57 AM, Arvid Warnecke wrote:

> Hello,
>
> On Mon, Jul 09, 2012 at 09:10:12PM +0300, Asaf Mesika wrote:
>> On Jul 9, 2012, at 21:00 PM, Harsh J wrote:
>>> The hbase-daemon.sh does not ssh back into the host, so preserves any
>>> environment variables you haven't otherwise set in the hbase-env.sh
>>> file. I guess that did the trick for you.
>>>
>> Maybe you should look at the content of the jvm argument switch
>> -Djava.library.path, (ps -ef | grep hbase , to see the command line).
>> This will give you a hint on the directories the .so object is being
>> looked for.
>>
> It seems that that switch is only in the 'hbase' script itself. But
> something like that must be the difference, because in my shell I only
> set $HADOOP_HOME and $HBASE_HOME and $HADOOP_CLASSPATH via ~/.zshrc.
It's not only there. Since inside hbase file, you see the following bash section which pulls the value of java.library.path from the hadoop shell script:

#If avail, add Hadoop to the CLASSPATH and to the JAVA_LIBRARY_PATH
HADOOP_IN_PATH=$(PATH="${HADOOP_HOME:-${HADOOP_PREFIX}}/bin:$PATH" which hadoop 2>/dev/null)
if [ -f ${HADOOP_IN_PATH} ]; then
  HADOOP_JAVA_LIBRARY_PATH=$(HADOOP_CLASSPATH="$CLASSPATH" ${HADOOP_IN_PATH} \
                             org.apache.hadoop.hbase.util.GetJavaProperty java.library.path 2>/dev/null)
  if [ -n "$HADOOP_JAVA_LIBRARY_PATH" ]; then
    JAVA_LIBRARY_PATH=$(append_path "${JAVA_LIBRARY_PATH}" "$HADOOP_JAVA_LIBRARY_PATH")
  fi
  CLASSPATH=$(append_path "${CLASSPATH}" `${HADOOP_IN_PATH} classpath 2>/dev/null`)
fi

if [ -d "${HBASE_HOME}/build/native" -o -d "${HBASE_HOME}/lib/native" ]; then
  if [ -z $JAVA_PLATFORM ]; then
    JAVA_PLATFORM=`CLASSPATH=${CLASSPATH} ${JAVA} org.apache.hadoop.util.PlatformName | sed -e "s/ /_/g"`
  fi
  if [ -d "$HBASE_HOME/build/native" ]; then
    JAVA_LIBRARY_PATH=$(append_path "$JAVA_LIBRARY_PATH" ${HBASE_HOME}/build/native/${JAVA_PLATFORM}/lib)
  fi

  if [ -d "${HBASE_HOME}/lib/native" ]; then
    JAVA_LIBRARY_PATH=$(append_path "$JAVA_LIBRARY_PATH" ${HBASE_HOME}/lib/native/${JAVA_PLATFORM})
  fi
fi
>
> Cheers,
> Arvid
>
> PS: Too bad that all those handy scripts have been removed in CDH4 completly
> btw. Now you have to send the output to a log file by yourself.
>
>
> --
> [ Arvid Warnecke ][ arvid (at) nostalgix (dot) org ]
> [ IRC/OPN: "madhatter" ][ http://www.nostalgix.org ]
> ---[  ThreePiO was right: Let the Wookiee win.  ]---