Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> execute hadoop job from remote web application


Copy link to this message
-
Re: execute hadoop job from remote web application
Oleg
      If you are looking at how to submit your jobs using JobClient then the
below sample can give you a start.

//get the configuration parameters and assigns a job name
        JobConf conf = new JobConf(getConf(), MyClass.class);
        conf.setJobName("SMS Reports");

        //setting key value types for mapper and reducer outputs
        conf.setOutputKeyClass(Text.class);
        conf.setOutputValueClass(Text.class);

        //specifying the custom reducer class
        conf.setReducerClass(SmsReducer.class);

        //Specifying the input directories(@ runtime) and Mappers
independently for inputs from multiple sources
        FileInputFormat.addInputPath(conf, new Path(args[0]));

        //Specifying the output directory @ runtime
        FileOutputFormat.setOutputPath(conf, new Path(args[1]));

        JobClient.runJob(conf);

Along with the hadoop jars you may need to have the config files as well on
your client.

The sample is from old map reduce API. You can use the new one as well in
that we use the Job instead of JobClient.

Hope it helps!..

Regards
Bejoy.K.S
On Tue, Oct 18, 2011 at 5:00 PM, Oleg Ruchovets <[EMAIL PROTECTED]>wrote:

> Excellent. Can you give a small example of code.
>
>
> On Tue, Oct 18, 2011 at 1:13 PM, Uma Maheswara Rao G 72686 <
> [EMAIL PROTECTED]> wrote:
>
> >
> > ----- Original Message -----
> > From: Oleg Ruchovets <[EMAIL PROTECTED]>
> > Date: Tuesday, October 18, 2011 4:11 pm
> > Subject: execute hadoop job from remote web application
> > To: [EMAIL PROTECTED]
> >
> > > Hi , what is the way to execute hadoop job on remote cluster. I
> > > want to
> > > execute my hadoop job from remote web  application , but I didn't
> > > find any
> > > hadoop client (remote API) to do it.
> > >
> > > Please advice.
> > > Oleg
> > >
> > You can put the Hadoop jars in your web applications classpath and find
> the
> > Class JobClient and submit the jobs using it.
> >
> > Regards,
> > Uma
> >
> >
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB