Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> Trigger job from Java application causes ClassNotFound


+
Steve Armstrong 2012-07-26, 23:18
+
syed kather 2012-07-27, 01:10
+
Steve Armstrong 2012-07-27, 01:20
Copy link to this message
-
Re: Trigger job from Java application causes ClassNotFound
Hi Steve

But if I try and run it from my dev pc in Eclipse (where all the
same dependencies are still in the classpath), and add the 3 hadoop
xml files to the classpath, it triggers hadoop jobs, but they fail
with error

There is problem in eclipse build path . I had faced same problem when i am
trying to do clustering . Have look on build path.
In mvn case it will download all the dependency jar from repo . if you want
to excete that in eclipse then you have to configure build path

I can suggest you to look at
http://shuyo.wordpress.com/2011/02/01/mahout-development-environment-with-maven-and-eclipse-1/

which can help you ..

            Thanks and Regards,
        S SYED ABDUL KATHER

On Fri, Jul 27, 2012 at 6:51 AM, Steve Armstrong [via Lucene] <
ml-node+[EMAIL PROTECTED]> wrote:

> Hi Syed,
>
> Do you mean I need to deploy the mahout jars to the lib directory of
> the master node? Or all the data nodes? Or is there a way to simply
> tell the hadoop job launcher to upload the jars itself?
>
> Steve
>
> On Thu, Jul 26, 2012 at 6:10 PM, syed kather <[hidden email]<http://user/SendEmail.jtp?type=node&node=3997615&i=0>>
> wrote:
>
> > Hi Steve ,
> > I hope you had missed that Sep ific jar to copy into your Hadoop lib
> > directories.  Have a look on ur lib .
> > On Jul 27, 2012 4:49 AM, "Steve Armstrong" <[hidden email]<http://user/SendEmail.jtp?type=node&node=3997615&i=1>>
> wrote:
> >
> >> Hello,
> >>
> >> I'm trying to trigger a Mahout job from inside my Java application
> >> (running in Eclipse), and get it running on my cluster. I have a main
> >> class that simply contains:
> >>
> >> String[] args = new String[] { "--input", "/input/triples.csv",
> >> "--output", "/output/vectors.txt", "--similarityClassname",
> >> VectorSimilarityMeasures.SIMILARITY_COOCCURRENCE.toString(),
> >> "--numRecommendations", "10000", "--tempDir", "temp/" +
> >> System.currentTimeMillis() };
> >> Configuration conf = new Configuration();
> >> ToolRunner.run(conf, new RecommenderJob(), args);
> >>
> >> If I package the whole project up in a single jar (using Maven), copy
> >> it to the namenode, and run it with "hadoop jar project.jar" it works
> >> fine. But if I try and run it from my dev pc in Eclipse (where all the
> >> same dependencies are still in the classpath), and add the 3 hadoop
> >> xml files to the classpath, it triggers hadoop jobs, but they fail
> >> with errors like:
> >>
> >> 12/07/26 14:42:09 INFO mapred.JobClient: Task Id :
> >> attempt_201206261211_0173_m_000001_0, Status : FAILED
> >> Error: java.lang.ClassNotFoundException:
> com.google.common.primitives.Longs
> >>         at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
> >>         at java.security.AccessController.doPrivileged(Native Method)
> >>         at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
> >>         at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
> >> ...
> >>
> >> What I'm trying to create is a self-contained JAR that can be run from
> >> the command-line and launch the mahout job on the cluster. I've got
> >> this all working with embedded pig scripts, but I can't get it working
> >> here.
> >>
> >> Any help is appreciated, or advice on better ways to trigger the jobs
> from
> >> code.
> >>
> >> Thanks
> >>
>
>
> ------------------------------
>  If you reply to this email, your message will be added to the discussion
> below:
>
> http://lucene.472066.n3.nabble.com/Trigger-job-from-Java-application-causes-ClassNotFound-tp3997583p3997615.html
>  To unsubscribe from Lucene, click here<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=472066&code=aW4uYWJkdWxAZ21haWwuY29tfDQ3MjA2NnwxMDczOTUyNDEw>
> .
> NAML<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
THANKS AND REGARDS,
SYED ABDUL KATHER
View this message in context: http://lucene.472066.n3.nabble.com/Trigger-job-from-Java-application-causes-ClassNotFound-tp3997583p3997686.html
Sent from the Hadoop lucene-users mailing list archive at Nabble.com.
+
John Armstrong 2012-07-27, 11:36
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB