Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Accumulo >> mail # user >> pragmattically invoking map reduce job from a .war deployed in tomcat


+
Cornish, Duane C. 2013-02-08, 17:25
Copy link to this message
-
Re: pragmattically invoking map reduce job from a .war deployed in tomcat
On Fri, Feb 8, 2013 at 12:25 PM, Cornish, Duane C. <[EMAIL PROTECTED]
> wrote:

> I have an accumulo instance running on a small 4 node cluster.  The
> namenode is running tomcat.  I am using a restful service to allow clients
> to access accumulo and kick off map reduce jobs.****
>
> ** **
>
> Previously, I had been creating runnable jars of my map reduce jobs.  The
> restful service would make a command line call to run the jar.  i.e. “java
> –jar mapreducejob.java –arg1 –arg2 etc.”  ****
>
> ** **
>
> I am now trying to switch these jobs to be called programmatically.  I am
> aware that I need to pass a comma separated list of jars into the
> ToolRunner.run call.  i.e.  ****
>
> ** **
>
> arg[0] = “-libjar”;
>

This should be -libjars.  However, that is probably not the issue because
the job classes themselves are not passed in via libjars.  Searching
around, I saw some indication that you are supposed to have your job jar
and other dependencies in the lib folder for your webapp.  Are you doing
that?

Billie

> ****
>
> arg[1] = “accumulo-core-1.4.1,jar2” etc.****
>
> arg[2] = “myArg1”;****
>
> arg[3] = “myArg2”;****
>
> etc.****
>
> ** **
>
> When I run my job, I get an error in in my map tasks in the jobtracker
> Admin webpage:****
>
> ** **
>
> java.lang.RuntimeException: java.lang.ClassNotFoundException: <package.Name>.<functionNameOfMyMapper>  ****
>
> ** **
>
> I don’t know why the system can’t find the mapper function.  It is in the
> same package as my Job file and they should all be contained in the .war
> file that is deployed in tomcat.  ****
>
> ** **
>
> Has anyone had success calling a map reduce job programmatically from
> within tomcat?****
>
> ** **
>
> Thanks,****
>
> Duane****
>
+
Cornish, Duane C. 2013-02-11, 15:10
+
Bob.Thorman@... 2013-02-08, 19:26
+
Cornish, Duane C. 2013-02-11, 15:14
+
Bob.Thorman@... 2013-02-11, 16:20
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB