On Fri, Feb 8, 2013 at 10:34 AM, David Medinets <[EMAIL PROTECTED]>wrote:
> I am running a map-reduce job. As soon as my mapper tried to serialize
> a Mutation I run into a NoSuchMethodError in reference to
> FieldValueMetaData. I could simply delete the hue-plugins jar file but
> that seems inelegant to me. When running a mapper can I shift when the
> jar files are loaded? Would HADOOP_USER_CLASSPATH_FIRST help in this
> situation? What about adding the libthift jar file to the
> sun.boot.class.path property?
>
I found a likely answer by searching for "hadoop distributedcache
precedence" which located p. 163 of Hadoop: The Definitive Guide. It
mentions this issue specifically under "Task Classpath Precedence." On the
client side, you can force Hadoop to put the user classpath first by
setting HADOOP_USER_CLASSPATH_FIRST to true. However, for a task
classpath, you should set "mapreduce.task.classpath.first" to true. A bit
more searching leads me to believe that this property name is different in
different versions of Hadoop. I just checked 1.0.3 and it appears to be
"mapreduce.user.classpath.first" there. This question indicates there
could be at least two other possible values:
http://stackoverflow.com/questions/11685949/overriding-default-hadoop-jars-in-class-pathApparently you should check the static property names in
org.apache.hadoop.mapred.TaskRunner to find the appropriate property name
for your version of Hadoop. Let us know if that works!
Billie