Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # dev >> Re: hadoop native libs 32 and 64 bit


Copy link to this message
-
Re: hadoop native libs 32 and 64 bit
I don't think the latter's been explained or done. Besides, I think
the latter is related more to packaging code (RPMs, etc.) than source
tarballs.

We ship source tarballs in Apache Hadoop releases, and Apache Bigtop
ships/will ship packaged artifacts (going forward). We don't currently
guarantee the availability of properly working precompiled native
artifacts for any platform today, but we can make a note to state so.

For instance, some of the current natives such as snappy fail to work
on CentOS5 if compiled on CentOS6, etc., so there's nothing we can do
aside of shipping sources with commands on building your own natives.
Or better, rely on Apache Bigtop to provide proper packages/package
building scripts?

On Mon, Aug 27, 2012 at 11:41 PM, Steven Willis <[EMAIL PROTECTED]> wrote:
> (moving to common-dev)
>
> Thanks Harsh,
>
> So what's the final outcome of these changes? Do we get both 32 and 64 bit libraries in the release tarball? Will they be underneath an arch dir, or directly under lib/native? I'm just a bit confused because the issue you reference is:
>
> "native libs should be under lib/native/ dir"
> https://issues.apache.org/jira/browse/HADOOP-7874
>
> but there's also this one:
>
> "change location of the native libraries to lib instead of lib/native"
> https://issues.apache.org/jira/browse/HADOOP-7996
>
> -Steven Willis
>
>> -----Original Message-----
>> From: Harsh J [mailto:[EMAIL PROTECTED]]
>> Sent: Monday, August 27, 2012 1:17 PM
>> To: [EMAIL PROTECTED]
>> Subject: Re: hadoop native libs 32 and 64 bit
>>
>> Hi Steven,
>>
>> You may also use the common-dev@ lists for development
>> discussions/issues around common elements :)
>>
>> Just for some context, this was changed in 2.x by us via:
>> https://issues.apache.org/jira/browse/HADOOP-7874
>>
>> On Mon, Aug 27, 2012 at 10:39 PM, Steven Willis <[EMAIL PROTECTED]>
>> wrote:
>> > Hi,
>> >
>> > I've been looking for both the 32 and 64 bit hadoop native libraries
>> and it looks like the existence and location of these libraries keeps
>> changing between releases. I downloaded the following releases:
>> >
>> > hadoop-0.22.0
>> > hadoop-0.23.0
>> > hadoop-0.23.1
>> > hadoop-1.0.1
>> > hadoop-1.0.2
>> > hadoop-1.0.3
>> > hadoop-2.0.0-alpha
>> > hadoop-2.0.1-alpha
>> >
>> > and looked for the location of 'libhadoop.so*' and ran 'file' on any
>> results, this is what I found:
>> >
>> > ./hadoop-0.22.0/lib/native/Linux-amd64-64/libhadoop.so:       ELF 64-
>> bit LSB shared object
>> > ./hadoop-0.22.0/lib/native/Linux-amd64-64/libhadoop.so.1:     ELF 64-
>> bit LSB shared object
>> > ./hadoop-0.22.0/lib/native/Linux-amd64-64/libhadoop.so.1.0.0: ELF 64-
>> bit LSB shared object
>> > ./hadoop-0.23.0/lib/libhadoop.so:
>> symbolic link to `libhadoop.so.1.0.0'
>> > ./hadoop-0.23.0/lib/libhadoop.so.1:
>> symbolic link to `libhadoop.so.1.0.0'
>> > ./hadoop-0.23.0/lib/libhadoop.so.1.0.0:                       ELF 32-
>> bit LSB shared object
>> > ./hadoop-0.23.1/lib/native/libhadoop.so:
>> symbolic link to `libhadoop.so.1.0.0'
>> > ./hadoop-0.23.1/lib/native/libhadoop.so.1:
>> symbolic link to `libhadoop.so.1.0.0'
>> > ./hadoop-0.23.1/lib/native/libhadoop.so.1.0.0:                ELF 32-
>> bit LSB shared object
>> > ./hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.so:        ELF 64-
>> bit LSB shared object
>> > ./hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.so.1:      ELF 64-
>> bit LSB shared object
>> > ./hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.so.1.0.0:  ELF 64-
>> bit LSB shared object
>> > ./hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.so:         ELF 32-
>> bit LSB shared object
>> > ./hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.so.1:       ELF 32-
>> bit LSB shared object
>> > ./hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.so.1.0.0:   ELF 32-
>> bit LSB shared object
>> > ./hadoop-1.0.2/lib/native/Linux-amd64-64/libhadoop.so:        ELF 64-
>> bit LSB shared object
>> > ./hadoop-1.0.2/lib/native/Linux-amd64-64/libhadoop.so.1:      ELF 64-

Harsh J