Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> execute hadoop job from remote web application


Copy link to this message
-
Re: execute hadoop job from remote web application
On 18/10/11 17:56, Harsh J wrote:
> Oleg,
>
> It will pack up the jar that contains the class specified by
> "setJarByClass" into its submission jar and send it up. Thats the
> function of that particular API method. So, your deduction is almost
> right there :)
>
> On Tue, Oct 18, 2011 at 10:20 PM, Oleg Ruchovets<[EMAIL PROTECTED]>  wrote:
>> So you mean that in case I am going to submit job remotely and
>> my_hadoop_job.jar
>> will be in class path of my web application it will submit job with
>> my_hadoop_job.jar to
>> remote hadoop machine (cluster)?
>>
There's also the problem of waiting for your work to finish. If you want
to see something complicated that does everything but JAR upload, I have
some code here that listens for events coming out of the job and so
builds up a history of what is happening. It also does better preflight
checking of source and dest data directories

http://smartfrog.svn.sourceforge.net/viewvc/smartfrog/trunk/core/hadoop-components/hadoop-ops/src/org/smartfrog/services/hadoop/mapreduce/submitter/SubmitterImpl.java
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB