Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce, mail # user - Maven dependencies for LocalJobRunner?

Copy link to this message
Maven dependencies for LocalJobRunner?
Matthias Friedrich 2012-10-07, 07:38

when running a simple word count example with Hadoop 1.0.3 and
2.0.1-alpha from the IDE (that is, without "hadoop jar"), I get
ClassNotFoundExceptions because LocalJobRunner needs dependencies
that are not referenced from hadoop-client's POM. For 1.0.3 it's
commons-cli and commons-httpclient, and for 2.0.1-alpha it's
commons-cli, commons-httpclient, and avro.

Things work fine when I add them, but I think since these libraries
are part of Hadoop's runtime classpath they should also be referenced
from the POM. Otherwise users might add them in different versions to
their classpath, which may lead to code breaking on the cluster that
worked fine locally.

So, question is: Is this a bug in Hadoop's POMs or is there something
I'm overlooking?