Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Maven dependencies for LocalJobRunner?


Copy link to this message
-
Maven dependencies for LocalJobRunner?
Hi,

when running a simple word count example with Hadoop 1.0.3 and
2.0.1-alpha from the IDE (that is, without "hadoop jar"), I get
ClassNotFoundExceptions because LocalJobRunner needs dependencies
that are not referenced from hadoop-client's POM. For 1.0.3 it's
commons-cli and commons-httpclient, and for 2.0.1-alpha it's
commons-cli, commons-httpclient, and avro.

Things work fine when I add them, but I think since these libraries
are part of Hadoop's runtime classpath they should also be referenced
from the POM. Otherwise users might add them in different versions to
their classpath, which may lead to code breaking on the cluster that
worked fine locally.

So, question is: Is this a bug in Hadoop's POMs or is there something
I'm overlooking?

Thanks,
  Matthias
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB