Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Accumulo >> mail # user >> pragmattically invoking map reduce job from a .war deployed in tomcat


Copy link to this message
-
RE: pragmattically invoking map reduce job from a .war deployed in tomcat
I do this same thing, but during deployment of my war I also deploy my
job jars and their dependencies to $ACCUMULO_HOME\lib\ext to make sure
they are available to the ensemble.  

 

From: Cornish, Duane C. [mailto:[EMAIL PROTECTED]]
Sent: Friday, February 08, 2013 11:25
To: [EMAIL PROTECTED]
Subject: pragmattically invoking map reduce job from a .war deployed in
tomcat

 

I have an accumulo instance running on a small 4 node cluster.  The
namenode is running tomcat.  I am using a restful service to allow
clients to access accumulo and kick off map reduce jobs.

 

Previously, I had been creating runnable jars of my map reduce jobs.
The restful service would make a command line call to run the jar.  i.e.
"java -jar mapreducejob.java -arg1 -arg2 etc."  

 

I am now trying to switch these jobs to be called programmatically.  I
am aware that I need to pass a comma separated list of jars into the
ToolRunner.run call.  i.e.  

 

arg[0] = "-libjar";

arg[1] = "accumulo-core-1.4.1,jar2" etc.

arg[2] = "myArg1";

arg[3] = "myArg2";

etc.

 

When I run my job, I get an error in in my map tasks in the jobtracker
Admin webpage:

 

java.lang.RuntimeException: java.lang.ClassNotFoundException:
<package.Name>.<functionNameOfMyMapper>  

 

I don't know why the system can't find the mapper function.  It is in
the same package as my Job file and they should all be contained in the
.war file that is deployed in tomcat.  

 

Has anyone had success calling a map reduce job programmatically from
within tomcat?

 

Thanks,

Duane