Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - Building LZO on hadoop


Copy link to this message
-
Re: Building LZO on hadoop
Saptarshi Guha 2009-04-01, 19:55
Fixed. In the configure script src/native/
change
 echo 'int main(int argc, char **argv){return 0;}' > conftest.c
  if test -z "`${CC} ${LDFLAGS} -o conftest conftest.c -llzo2 2>&1`"; then
        if test ! -z "`which objdump | grep -v 'no objdump'`"; then
      ac_cv_libname_lzo2="`objdump -p conftest | grep NEEDED | grep
lzo2 | sed 's/\W*NEEDED\W*\(.*\)\W*$/\"\1\"/'`"
    elif test ! -z "`which ldd | grep -v 'no ldd'`"; then
      ac_cv_libname_lzo2="`ldd conftest | grep lzo2 | sed
's/^[^A-Za-z0-9]*\([A-Za-z0-9\.]*\)[^A-Za-z0-9]*=>.*$/\"\1\"/'`"
    else
      { { echo "$as_me:$LINENO: error: Can't find either 'objdump' or
'ldd' to compute the dynamic library for '-llzo2'" >&5
echo "$as_me: error: Can't find either 'objdump' or 'ldd' to compute
the dynamic library for '-llzo2'" >&2;}
   { (exit 1); exit 1; }; }
    fi
  else
    ac_cv_libname_lzo2=libnotfound.so
  fi
  rm -f conftest*

lzo2 to lzo.so.2 (again this depends on what the user has), also set
CFLAGS and LDFLAGS to include your lzo libs/incs

Saptarshi Guha

On Wed, Apr 1, 2009 at 2:29 PM, Saptarshi Guha <[EMAIL PROTECTED]> wrote:
> I checked out hadoop-core-0.19
> export CFLAGS=$CUSTROOT/include
> export LDFLAGS=$CUSTROOT/lib
>
> (they contain lzo which was built with --shared)
>>ls $CUSTROOT/include/lzo/
> lzo1a.h  lzo1b.h  lzo1c.h  lzo1f.h  lzo1.h  lzo1x.h  lzo1y.h  lzo1z.h
> lzo2a.h  lzo_asm.h  lzoconf.h  lzodefs.h  lzoutil.h
>
>>ls $CUSTROOT/lib/
> liblzo2.so  liblzo.a  liblzo.la  liblzo.so  liblzo.so.1  liblzo.so.2
> liblzo.so.2.0.0
>
> I then run (from hadoop-core-0.19.1/)
> ant -Dcompile.native=true
>
> I get messages like : (many others like this)
> exec] configure: WARNING: lzo/lzo1x.h: accepted by the compiler,
> rejected by the preprocessor!
>     [exec] configure: WARNING: lzo/lzo1x.h: proceeding with the
> compiler's result
>     [exec] checking for lzo/lzo1x.h... yes
>     [exec] checking Checking for the 'actual' dynamic-library for
> '-llzo2'... (cached)
>     [exec] checking lzo/lzo1y.h usability... yes
>     [exec] checking lzo/lzo1y.h presence... no
>     [exec] configure: WARNING: lzo/lzo1y.h: accepted by the compiler,
> rejected by the preprocessor!
>     [exec] configure: WARNING: lzo/lzo1y.h: proceeding with the
> compiler's result
>     [exec] checking for lzo/lzo1y.h... yes
>     [exec] checking Checking for the 'actual' dynamic-library for
> '-llzo2'... (cached)
>
> and finally,
> ive/src/org/apache/hadoop/io/compress/lzo/LzoCompressor.c  -fPIC -DPIC
> -o .libs/LzoCompressor.o
>     [exec] /ln/meraki/custom/hadoop-core-0.19.1/src/native/src/org/apache/hadoop/io/compress/lzo/LzoCompressor.c:
> In function 'Java_org_apache_hadoop_io_compress_lzo_LzoCompressor_initIDs':
>     [exec] /ln/meraki/custom/hadoop-core-0.19.1/src/native/src/org/apache/hadoop/io/compress/lzo/LzoCompressor.c:137:
> error: expected expression before ',' token
>
>
> Any ideas?
> Saptarshi Guha
>