Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop, mail # user - Unable to load native hadoop library


Copy link to this message
-
Re: Unable to load native hadoop library
Brenden Cobb 2014-01-06, 16:50
I did not have the correct path to hadoop in my sqoop conf file. Once I fixed that everything was ok.

-BC

From: Abraham Elmahrek <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Reply-To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Date: Sunday, January 5, 2014 8:10 PM
To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Subject: Re: Unable to load native hadoop library

Brenden,

Happy to hear things are working. Would you mind posting back what you did to get around your latest issue? That way, if someone else runs into this problem and they start searching around... they can refer back to this post.

-Abe
On Sun, Jan 5, 2014 at 1:00 PM, Brenden Cobb <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
Nevermind! A little more tinkering and I got the basics working. Thanks again for the tip.

-BC
From: Brenden Cobb <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Reply-To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Date: Sunday, January 5, 2014 3:01 PM

To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Subject: Re: Unable to load native hadoop library

Thanks for the tip Abe. I've set bcobb-mbpr.local to 127.0.0.1 and now end up with:

14/01/05 14:51:49 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId14/01/05 14:51:49 ERROR security.UserGroupInformation: PriviledgedActionException as:bcobb (auth:SIMPLE) cause:java.io.IOException: Failed on local exception: java.io.EOFException; Host Details : local host is: "localhost/127.0.0.1<http://127.0.0.1>"; destination host is: "localhost":8020;
14/01/05 14:51:49 ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: Failed on local exception: java.io.EOFException; Host Details : local host is: "localhost/127.0.0.1<http://127.0.0.1>"; destination host is: "localhost":8020;
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:764)
Any tips?

Thanks

-BC

From: Abraham Elmahrek <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Reply-To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Date: Thursday, January 2, 2014 11:21 PM
To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Subject: Re: Unable to load native hadoop library

It seems like your localhost "bcobb-mbpr.local" doesn't loop back. Try using "localhost" in your configs instead or add "bcobb-mbpr.local" to your /etc/hosts file.

-Abe
On Thu, Jan 2, 2014 at 7:30 PM, rudra tripathy <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
Hi Cobb,

Try adding jars to classpath and path and check whether it is working.

Thanks and Regards,
Rudra
--------------------------------------------
On Fri, 3/1/14, Brenden Cobb <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:

 Subject: Unable to load native hadoop library
 To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
 Date: Friday, 3 January, 2014, 3:37 AM
 Hi-
 I'm just getting started with Sqoop and am running
 some tests against mySQL to get the hang of things.
 Sqoop:  1.4.4
 hadoop: tried both 1.2.1 and 2.2.0
 platform: OS X

 Unfortunately I am hitting the following error when
 trying to import. Appreciate any help!

 -BC
 ==> sqoop import --connect
 jdbc:mysql://localhost/hadoopguide --table widgets -m
 1

 14/01/02 15:57:58 INFO manager.MySQLManager: Preparing
 to use a MySQL streaming resultset.
 14/01/02 15:57:58 INFO tool.CodeGenTool: Beginning code
 generation
 14/01/02 15:57:59 INFO manager.SqlManager: Executing
 SQL statement: SELECT t.* FROM `widgets` AS t LIMIT 1
 14/01/02 15:57:59 INFO manager.SqlManager: Executing
 SQL statement: SELECT t.* FROM `widgets` AS t LIMIT 1
 14/01/02 15:57:59 INFO orm.CompilationManager:
 HADOOP_MAPRED_HOME is /usr/local
 Note:
 /tmp/sqoop-bcobb/compile/d54504715f779b2aa610a608f51aa56b/widgets.java
 uses or overrides a deprecated API.
 Note: Recompile with -Xlint:deprecation for
 details.
 14/01/02 15:58:00 INFO orm.CompilationManager: Writing
 jar file:
 /tmp/sqoop-bcobb/compile/d54504715f779b2aa610a608f51aa56b/widgets.jar
 ..
 14/01/02 15:58:00 INFO mapreduce.ImportJobBase:
 Beginning import of widgets
 2014-01-02 15:58:00.683 java[60905:1903] Unable to load
 realm info from SCDynamicStore
 ..

 4/01/02 15:58:05 WARN util.NativeCodeLoader: Unable to
 load native-hadoop library for your platform... using
 builtin-java classes where applicable

 14/01/02 15:58:11 INFO mapred.JobClient: Cleaning up
 the staging area
 file:/tmp/hadoop-bcobb/mapred/staging/bcobb678313846/.staging/job_local678313846_0001
 14/01/02 15:58:11 ERROR security.UserGroupInformation:
 PriviledgedActionException as:bcobb
 cause:java.net.UnknownHostException: bcobb-mbpr.local:
 bcobb-mbpr.local: nodename nor servname provided, or not
 known
 14/01/02 15:58:11 ERROR tool.ImportTool: Encountered
 IOException running import job:
 java.net.UnknownHostException: bcobb-mbpr.local:
 bcobb-mbpr.local: nodename nor servname provided, or not
 known
 at
 java.net.InetAddress.getLocalHost(InetAddress.java:1473)
 at
 org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:960)
 at
 org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:936)
 at
 java.security.AccessController.doPrivileged(Native
 Method)
 at
 javax.security.auth.Subject.doAs(Subject.java:415)
 at
 org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
 at
 org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:936)
 at
 org.apache.hadoop.mapreduce.Job.submit(Job.java:550)
 at
 org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:580)
 at
 org.apache.sqoop.mapreduce.Imp