Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> JNI native library loading problem in standalone mode


Copy link to this message
-
JNI native library loading problem in standalone mode
Hi, I am trying to run a JNI application on StandAlone mode and I have been
getting this result ever since.
I've looked up every possible webs and sites but never could get the
solution to it.
Please help me find out what's wrong.

I have 2 java files (LangAnal.java, freeparser.java)
LangAnal.java belongs to a package "org.etri.mp2893"
freeparser.java belongs to a package "org.knu.freeparser"

freeparser.java loads "libetri_lmi.so" which is in "/home/qa/lang/dll4jni/"
and I believe "libetri_lmi.so" uses another library "libparser.so" which is
in "/home/qa/lang/lib".

I have compiled LangAnal.java and freeparser.java. The resulting classes are
placed respectively at "/home/qa/lang/dll4jni/org/etri/mp2893/" and
"/home/qa/lang/dll4jni/org/knu/freeparser/".

Then I have archived those classes by

"jar -cvef org.etri.mp2893.LangAnal ./LangAnal.jar ./org/*"

from the directory of "/home/qa/lang/dll4jni/".

Then I tried to run

hadoop jar LangAnal.jar ./news.txt ./output

from the same directory and I get the result down below.

I tried to set LD_LIBRARY_PATH to
".:/home/qa/lang/dll4jni:/home/qa/lang/lib"
I tried java.setProperty("java.library.path", "/home/qa/lang/dll4jni"); in
freeparser.java.
So far nothing has worked.

But the funny thing is, if I run

hadoop -Djava.library.path=/home/qa/lang/dll4jni org.etri.mp2893.LangAnal
./news.txt ./output

from "/home/qa/lang/dll4jni/" it works just fine. So I guess the problem is
just about jar file.
And I intend to run this application in Fully-Distributed mode soon, I have
to figure out how to run jar files.

Please someone help me.

-------------------------------------------------------------------------------------------------------------------

[qa@qa128 dll4jni]$ hadoop jar LangAnal.jar ./news.txt ./output
10/06/01 01:19:39 INFO jvm.JvmMetrics: Initializing JVM Metrics with
processName=JobTracker, sessionId10/06/01 01:19:39 WARN mapred.JobClient: Use GenericOptionsParser for
parsing the arguments. Applications should implement Tool for the same.
10/06/01 01:19:39 WARN mapred.JobClient: No job jar file set.  User classes
may not be found. See JobConf(Class) or JobConf#setJar(String).
10/06/01 01:19:39 INFO input.FileInputFormat: Total input paths to process :
1
10/06/01 01:19:39 INFO mapred.JobClient: Running job: job_local_0001
10/06/01 01:19:39 INFO input.FileInputFormat: Total input paths to process :
1
10/06/01 01:19:39 WARN mapred.LocalJobRunner: job_local_0001
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:569)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:113)
... 3 more
Caused by: java.lang.UnsatisfiedLinkError: no etri_lmi in java.library.path
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1709)
at java.lang.Runtime.loadLibrary0(Runtime.java:823)
at java.lang.System.loadLibrary(System.java:1030)
at org.knu.freeparser.freeparser.<clinit>(freeparser.java:9)
at org.etri.mp2893.LangAnal$LangAnalMapper.<init>(LangAnal.java:26)
... 8 more
10/06/01 01:19:40 INFO mapred.JobClient:  map 0% reduce 0%
10/06/01 01:19:40 INFO mapred.JobClient: Job complete: job_local_0001
10/06/01 01:19:40 INFO mapred.JobClient: Counters: 0
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB