Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - Submitting and running hadoop jobs Programmatically


Copy link to this message
-
Re: Submitting and running hadoop jobs Programmatically
madhu phatak 2011-07-27, 10:16
Thank you . Will have a look on it.

On Wed, Jul 27, 2011 at 3:28 PM, Steve Loughran <[EMAIL PROTECTED]> wrote:

> On 27/07/11 05:55, madhu phatak wrote:
>
>> Hi
>> I am submitting the job as follows
>>
>> java -cp
>>  Nectar-analytics-0.0.1-**SNAPSHOT.jar:/home/hadoop/**
>> hadoop-for-nectar/hadoop-0.21.**0/conf/*:$HADOOP_COMMON_HOME/**
>> lib/*:$HADOOP_COMMON_HOME/*
>> com.zinnia.nectar.regression.**hadoop.primitive.jobs.SigmaJob
>> input/book.csv
>> kkk11fffrrw 1
>>
>
> My code to submit jobs (via a declarative configuration) is up online
>
> http://smartfrog.svn.**sourceforge.net/viewvc/**
> smartfrog/trunk/core/hadoop-**components/hadoop-ops/src/org/**
> smartfrog/services/hadoop/**operations/components/**
> submitter/SubmitterImpl.java?**revision=8590&view=markup<http://smartfrog.svn.sourceforge.net/viewvc/smartfrog/trunk/core/hadoop-components/hadoop-ops/src/org/smartfrog/services/hadoop/operations/components/submitter/SubmitterImpl.java?revision=8590&view=markup>
>
> It's LGPL, but ask nicely and I'll change the header to Apache.
>
> That code doesn't set up the classpath by pushing out more JARs (I'm
> planning to push out .groovy scripts instead), but it can also poll for job
> completion, take a timeout (useful in small test runs), and do other things.
> I currently mainly use it for testing
>
>