Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> Enable Snappy compression - not able to load the libs on startup


+
Arvid Warnecke 2012-07-08, 07:48
+
Paul Cavallaro 2012-07-08, 16:45
+
Arvid Warnecke 2012-07-08, 20:21
+
Harsh J 2012-07-09, 01:44
+
Arvid Warnecke 2012-07-09, 17:44
+
Harsh J 2012-07-09, 18:00
+
Asaf Mesika 2012-07-09, 18:10
+
Arvid Warnecke 2012-07-10, 05:57
Copy link to this message
-
Re: Enable Snappy compression - not able to load the libs on startup

On Jul 10, 2012, at 8:57 AM, Arvid Warnecke wrote:

> Hello,
>
> On Mon, Jul 09, 2012 at 09:10:12PM +0300, Asaf Mesika wrote:
>> On Jul 9, 2012, at 21:00 PM, Harsh J wrote:
>>> The hbase-daemon.sh does not ssh back into the host, so preserves any
>>> environment variables you haven't otherwise set in the hbase-env.sh
>>> file. I guess that did the trick for you.
>>>
>> Maybe you should look at the content of the jvm argument switch
>> -Djava.library.path, (ps -ef | grep hbase , to see the command line).
>> This will give you a hint on the directories the .so object is being
>> looked for.
>>
> It seems that that switch is only in the 'hbase' script itself. But
> something like that must be the difference, because in my shell I only
> set $HADOOP_HOME and $HBASE_HOME and $HADOOP_CLASSPATH via ~/.zshrc.
It's not only there. Since inside hbase file, you see the following bash section which pulls the value of java.library.path from the hadoop shell script:

#If avail, add Hadoop to the CLASSPATH and to the JAVA_LIBRARY_PATH
HADOOP_IN_PATH=$(PATH="${HADOOP_HOME:-${HADOOP_PREFIX}}/bin:$PATH" which hadoop 2>/dev/null)
if [ -f ${HADOOP_IN_PATH} ]; then
  HADOOP_JAVA_LIBRARY_PATH=$(HADOOP_CLASSPATH="$CLASSPATH" ${HADOOP_IN_PATH} \
                             org.apache.hadoop.hbase.util.GetJavaProperty java.library.path 2>/dev/null)
  if [ -n "$HADOOP_JAVA_LIBRARY_PATH" ]; then
    JAVA_LIBRARY_PATH=$(append_path "${JAVA_LIBRARY_PATH}" "$HADOOP_JAVA_LIBRARY_PATH")
  fi
  CLASSPATH=$(append_path "${CLASSPATH}" `${HADOOP_IN_PATH} classpath 2>/dev/null`)
fi

if [ -d "${HBASE_HOME}/build/native" -o -d "${HBASE_HOME}/lib/native" ]; then
  if [ -z $JAVA_PLATFORM ]; then
    JAVA_PLATFORM=`CLASSPATH=${CLASSPATH} ${JAVA} org.apache.hadoop.util.PlatformName | sed -e "s/ /_/g"`
  fi
  if [ -d "$HBASE_HOME/build/native" ]; then
    JAVA_LIBRARY_PATH=$(append_path "$JAVA_LIBRARY_PATH" ${HBASE_HOME}/build/native/${JAVA_PLATFORM}/lib)
  fi

  if [ -d "${HBASE_HOME}/lib/native" ]; then
    JAVA_LIBRARY_PATH=$(append_path "$JAVA_LIBRARY_PATH" ${HBASE_HOME}/lib/native/${JAVA_PLATFORM})
  fi
fi
>
> Cheers,
> Arvid
>
> PS: Too bad that all those handy scripts have been removed in CDH4 completly
> btw. Now you have to send the output to a log file by yourself.
>
>
> --
> [ Arvid Warnecke ][ arvid (at) nostalgix (dot) org ]
> [ IRC/OPN: "madhatter" ][ http://www.nostalgix.org ]
> ---[  ThreePiO was right: Let the Wookiee win.  ]---
+
Arvid Warnecke 2012-07-10, 17:04
+
Asaf Mesika 2012-07-11, 18:07