Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> execute hadoop job from remote web application


Copy link to this message
-
Re: execute hadoop job from remote web application
On 18/10/11 17:56, Harsh J wrote:
> Oleg,
>
> It will pack up the jar that contains the class specified by
> "setJarByClass" into its submission jar and send it up. Thats the
> function of that particular API method. So, your deduction is almost
> right there :)
>
> On Tue, Oct 18, 2011 at 10:20 PM, Oleg Ruchovets<[EMAIL PROTECTED]>  wrote:
>> So you mean that in case I am going to submit job remotely and
>> my_hadoop_job.jar
>> will be in class path of my web application it will submit job with
>> my_hadoop_job.jar to
>> remote hadoop machine (cluster)?
>>
There's also the problem of waiting for your work to finish. If you want
to see something complicated that does everything but JAR upload, I have
some code here that listens for events coming out of the job and so
builds up a history of what is happening. It also does better preflight
checking of source and dest data directories

http://smartfrog.svn.sourceforge.net/viewvc/smartfrog/trunk/core/hadoop-components/hadoop-ops/src/org/smartfrog/services/hadoop/mapreduce/submitter/SubmitterImpl.java