Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # user - Re: Why do i get the following error all of a sudden?

Copy link to this message
Re: Why do i get the following error all of a sudden?
Michael Webster 2013-10-29, 19:31
I have seen this issue as well.  The problem seems to come down to an issue in
TableMapReduceUtil#findOrCreateJars.  This class loads dependency jars into
the distributed cache for MR jobs, this class in turn calls JarFinder#getJar
from hadoop.  The getJar method  returns a jar file containing all of the
dependency classes. If it can find the dependency as jar,
based on the protocol of the path URL, it just provides the path of the jar
file.  If it finds the class as a file it attempts to create a directory, by
default CWD/target/temp-dir, and build a jar there.  If the user submitting
the job doesn't have permissions to create that directory you get an
IOException.  Some possible solutions are:

1. chmod the directory you run the script from such that the user submitting
the job has permissions (i have verified that this works)
2. add a jar to the classpath containing the classes that you need,
that should cause JarFinder#getJar to just provide the path of the jar file
(I haven't yet tested this out yet, but it should work)

- michael