Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - Unable to Load Native-Hadoop Library for Your Platform


Copy link to this message
-
RE: Unable to Load Native-Hadoop Library for Your Platform
Uma Maheswara Rao G 2012-02-07, 10:04
Looks you are not using any compression in your code. Hadoop has some native libraries to load mainly for compression codecs.
When you want to use that compression tequniques, you need to compile with this compile.native option enable. Also need to set in java library path. If you are not using any such stuff, then you need not worry about that warning.
Please look at the below link for more information.
http://hadoop.apache.org/common/docs/current/native_libraries.html

Regards,
Uma
________________________________________
From: Bing Li [[EMAIL PROTECTED]]
Sent: Tuesday, February 07, 2012 3:08 PM
To: [EMAIL PROTECTED]
Subject: Unable to Load Native-Hadoop Library for Your Platform

Dear all,

I got an error when running a simple Java program on Hadoop. The program is
just to merge some local files to one and put it on Hadoop.

The code is as follows.

                ......

                Configuration conf = new Configuration();
                try
                {
                        FileSystem hdfs = FileSystem.get(conf);
                        FileSystem local = FileSystem.getLocal(conf);
                        Path inputDir = new Path("/home/libing/Temp/");
                        Path hdfsFile = new
Path("/tmp/user/libing/example.txt");

                        try
                        {
                                FileStatus[] inputFiles local.listStatus(inputDir);
                                FSDataOutputStream out hdfs.create(hdfsFile);
                                for (int i = 0; i < inputFiles.length; i ++)
                                {

System.out.println(inputFiles[i].getPath().getName());
                                        FSDataInputStream in local.open(inputFiles[i].getPath());
                                        byte buffer[] = new byte[256];
                                        int bytesRead = 0;
                                        while ((bytesRead in.read(buffer)) > 0)
                                        {
                                                out.write(buffer, 0,
bytesRead);
                                        }
                                        in.close();
                                }
                                out.close();
                        }
                        catch (IOException e)
                        {
                                e.printStackTrace();
                        }
                }
                catch (IOException e)
                {
                        e.printStackTrace();
                }

                ......

I run it with ant and got the following warning. BTW, all the relevant jar
packages from Hadoop are specified in the build.xml.

     [java] 2012-2-7 17:16:18 org.apache.hadoop.util.NativeCodeLoader
<clinit>
     [java] Warning: Unable to load native-hadoop library for your
platform... using builtin-java classes where applicable

The program got a correct result. But I cannot figure out what the above
problem is.

Thanks so much!
Bing