Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> hadoop native libs 32 and 64 bit


Copy link to this message
-
Re: hadoop native libs 32 and 64 bit
Hi Steven,

You may also use the common-dev@ lists for development
discussions/issues around common elements :)

Just for some context, this was changed in 2.x by us via:
https://issues.apache.org/jira/browse/HADOOP-7874

On Mon, Aug 27, 2012 at 10:39 PM, Steven Willis <[EMAIL PROTECTED]> wrote:
> Hi,
>
> I've been looking for both the 32 and 64 bit hadoop native libraries and it looks like the existence and location of these libraries keeps changing between releases. I downloaded the following releases:
>
> hadoop-0.22.0
> hadoop-0.23.0
> hadoop-0.23.1
> hadoop-1.0.1
> hadoop-1.0.2
> hadoop-1.0.3
> hadoop-2.0.0-alpha
> hadoop-2.0.1-alpha
>
> and looked for the location of 'libhadoop.so*' and ran 'file' on any results, this is what I found:
>
> ./hadoop-0.22.0/lib/native/Linux-amd64-64/libhadoop.so:       ELF 64-bit LSB shared object
> ./hadoop-0.22.0/lib/native/Linux-amd64-64/libhadoop.so.1:     ELF 64-bit LSB shared object
> ./hadoop-0.22.0/lib/native/Linux-amd64-64/libhadoop.so.1.0.0: ELF 64-bit LSB shared object
> ./hadoop-0.23.0/lib/libhadoop.so:                             symbolic link to `libhadoop.so.1.0.0'
> ./hadoop-0.23.0/lib/libhadoop.so.1:                           symbolic link to `libhadoop.so.1.0.0'
> ./hadoop-0.23.0/lib/libhadoop.so.1.0.0:                       ELF 32-bit LSB shared object
> ./hadoop-0.23.1/lib/native/libhadoop.so:                      symbolic link to `libhadoop.so.1.0.0'
> ./hadoop-0.23.1/lib/native/libhadoop.so.1:                    symbolic link to `libhadoop.so.1.0.0'
> ./hadoop-0.23.1/lib/native/libhadoop.so.1.0.0:                ELF 32-bit LSB shared object
> ./hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.so:        ELF 64-bit LSB shared object
> ./hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.so.1:      ELF 64-bit LSB shared object
> ./hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.so.1.0.0:  ELF 64-bit LSB shared object
> ./hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.so:         ELF 32-bit LSB shared object
> ./hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.so.1:       ELF 32-bit LSB shared object
> ./hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.so.1.0.0:   ELF 32-bit LSB shared object
> ./hadoop-1.0.2/lib/native/Linux-amd64-64/libhadoop.so:        ELF 64-bit LSB shared object
> ./hadoop-1.0.2/lib/native/Linux-amd64-64/libhadoop.so.1:      ELF 64-bit LSB shared object
> ./hadoop-1.0.2/lib/native/Linux-amd64-64/libhadoop.so.1.0.0:  ELF 64-bit LSB shared object
> ./hadoop-1.0.2/lib/native/Linux-i386-32/libhadoop.so:         ELF 32-bit LSB shared object
> ./hadoop-1.0.2/lib/native/Linux-i386-32/libhadoop.so.1:       ELF 32-bit LSB shared object
> ./hadoop-1.0.2/lib/native/Linux-i386-32/libhadoop.so.1.0.0:   ELF 32-bit LSB shared object
> ./hadoop-1.0.3/lib/native/Linux-amd64-64/libhadoop.so:        ELF 64-bit LSB shared object
> ./hadoop-1.0.3/lib/native/Linux-amd64-64/libhadoop.so.1:      ELF 64-bit LSB shared object
> ./hadoop-1.0.3/lib/native/Linux-amd64-64/libhadoop.so.1.0.0:  ELF 64-bit LSB shared object
> ./hadoop-1.0.3/lib/native/Linux-i386-32/libhadoop.so:         ELF 32-bit LSB shared object
> ./hadoop-1.0.3/lib/native/Linux-i386-32/libhadoop.so.1:       ELF 32-bit LSB shared object
> ./hadoop-1.0.3/lib/native/Linux-i386-32/libhadoop.so.1.0.0:   ELF 32-bit LSB shared object
> ./hadoop-2.0.0-alpha/lib/native/libhadoop.so:                 symbolic link to `libhadoop.so.1.0.0'
> ./hadoop-2.0.0-alpha/lib/native/libhadoop.so.1:               symbolic link to `libhadoop.so.1.0.0'
> ./hadoop-2.0.0-alpha/lib/native/libhadoop.so.1.0.0:           ELF 32-bit LSB shared object
> ./hadoop-2.0.1-alpha/lib/native/libhadoop.so:                 symbolic link to `libhadoop.so.1.0.0'
> ./hadoop-2.0.1-alpha/lib/native/libhadoop.so.1:               symbolic link to `libhadoop.so.1.0.0'
> ./hadoop-2.0.1-alpha/lib/native/libhadoop.so.1.0.0:           ELF 32-bit LSB shared object
>
> It looks like in 0.22.0, hadoop was only released with 64 bit libs. They were put in an arch directory under 'lib/native'. And there are copies for the .so files not symlinks. Hadoop 0.23.0 has only 32 bit libs which are directly under 'lib' (no 'native' or arch dir), and there are symlinks for the .so files. Hadoop 0.23.1 still has just 32 bit libs and symlinks, but they're now under 'lib/native'. From 1.0.1 to 1.0.3 we have both 32 and 64 bit libs and they're back under 'lib/native/ARCH', and we're back to having copies of the libs instead of symlinks. In 2.0.0-alpha and 2.0.1-alpha they go back to 32 bit only and under 'lib/native', and we get symlinks back.

Harsh J
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB