Hello,

I'm trying to find a way how to run hadoop MapReduce app from another
machine.
For instance I have *.jar file with MapReduce app it works ok when I run it
from command line for instance using this command: "hadoop jar
/usr/joe/wordcount.jar org.myorg.WordCount /usr/joe/wordcount/input
/usr/joe/wordcount/output"
But in situation when I have another server (simple web app) where user can
upload jar file, specify configuration for the MapReduce app and so on. And
this server should interact with hadoop server. I mean somehow to upload
this jar file to the hadoop server and run it with attributes.

So, right now I see only one way of how to do this, it's upload jar file to
the hadoop server and run remotely command: "hadoop jar
/usr/joe/wordcount.jar org.myorg.WordCount /usr/joe/wordcount/input
/usr/joe/wordcount/output".

So may be hadoop has spatial API for doing this kind of tasks remotely?
--
View this message in context: http://old.nabble.com/Run-hadoop-Map-Reduce-app-from-another-machine-tp32595264p32595264.html
Sent from the Hadoop core-user mailing list archive at Nabble.com.

Install hadoop on your local machine, copy the configuration files from the
remote
hadoop culuster server to your local machine(including the hosts file), then
you can
just submit a *.jar locally as before.
2011/10/5 oleksiy <[EMAIL PROTECTED]>

>
> Hello,
>
> I'm trying to find a way how to run hadoop MapReduce app from another
> machine.
> For instance I have *.jar file with MapReduce app it works ok when I run it
> from command line for instance using this command: "hadoop jar
> /usr/joe/wordcount.jar org.myorg.WordCount /usr/joe/wordcount/input
> /usr/joe/wordcount/output"
> But in situation when I have another server (simple web app) where user can
> upload jar file, specify configuration for the MapReduce app and so on. And
> this server should interact with hadoop server. I mean somehow to upload
> this jar file to the hadoop server and run it with attributes.
>
> So, right now I see only one way of how to do this, it's upload jar file to
> the hadoop server and run remotely command: "hadoop jar
> /usr/joe/wordcount.jar org.myorg.WordCount /usr/joe/wordcount/input
> /usr/joe/wordcount/output".
>
> So may be hadoop has spatial API for doing this kind of tasks remotely?
> --
> View this message in context:
> http://old.nabble.com/Run-hadoop-Map-Reduce-app-from-another-machine-tp32595264p32595264.html
> Sent from the Hadoop core-user mailing list archive at Nabble.com.
>
>

Hi Oleksiy,

No need to copy the jar file and execute the hadoop command remotely
to submit job.

Hadoop provides JobClient API to submit and monitor the jobs from remote
systems/remote application.

http://hadoop.apache.org/common/docs/r0.20.0/api/org/apache/hadoop/mapred/Jo
bClient.html

Here is an example on how to use JobClient to run the job  :
     // Create a new JobConf
     JobConf job = new JobConf(new Configuration(), MyJob.class);
    
     // Specify various job-specific parameters    
     job.setJobName("myjob");
    
     job.setInputPath(new Path("in"));
     job.setOutputPath(new Path("out"));
    
     job.setMapperClass(MyJob.MyMapper.class);
     job.setReducerClass(MyJob.MyReducer.class);

     // Submit the job, then poll for progress until the job is complete
     JobClient.runJob(job);

I hope this helps to solve problem.

Devaraj K

-----Original Message-----
From: oleksiy [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, October 05, 2011 3:32 PM
To: [EMAIL PROTECTED]
Subject: Run hadoop Map/Reduce app from another machine
Hello,

I'm trying to find a way how to run hadoop MapReduce app from another
machine.
For instance I have *.jar file with MapReduce app it works ok when I run it
from command line for instance using this command: "hadoop jar
/usr/joe/wordcount.jar org.myorg.WordCount /usr/joe/wordcount/input
/usr/joe/wordcount/output"
But in situation when I have another server (simple web app) where user can
upload jar file, specify configuration for the MapReduce app and so on. And
this server should interact with hadoop server. I mean somehow to upload
this jar file to the hadoop server and run it with attributes.

So, right now I see only one way of how to do this, it's upload jar file to
the hadoop server and run remotely command: "hadoop jar
/usr/joe/wordcount.jar org.myorg.WordCount /usr/joe/wordcount/input
/usr/joe/wordcount/output".

So may be hadoop has spatial API for doing this kind of tasks remotely?
--
View this message in context:
http://old.nabble.com/Run-hadoop-Map-Reduce-app-from-another-machine-tp32595
264p32595264.html
Sent from the Hadoop core-user mailing list archive at Nabble.com.