Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> RE: Submitting jobs programmatically (even from Windows, IDE...)


Copy link to this message
-
RE: Submitting jobs programmatically (even from Windows, IDE...)
Hi Vjeran,

First, thanks for the blog post!
As for the support for Windows, we are working actively to make Hadoop run smoothly on Windows.
The following JIRAs may provide some background and general outline of this effort.

https://issues.apache.org/jira/browse/HADOOP-8079
https://issues.apache.org/jira/browse/HADOOP-8562
https://issues.apache.org/jira/browse/MAPREDUCE-4401
https://issues.apache.org/jira/browse/YARN-191

You are very welcome to report any issues found on Windows as well.

Cheers,
Chuan
From: Vjeran Marcinko [mailto:[EMAIL PROTECTED]]
Sent: Monday, June 17, 2013 10:47 PM
To: [EMAIL PROTECTED]
Subject: Submitting jobs programmatically (even from Windows, IDE...)

Hi,

As a newcomer to Hadoop, I really struggled in the beginning to set things up with running jobs from my IDE, and after I dove deep into how things works, I discovered there are plenty of things that one has to take care, and many parameters that can influence one's mind on how to best deploy the jobs. Most of them I described in my blog with some additional utility code that can help many to submit their jobs, together with any necessary 3rd party libs:
http://pannoniancoder.blogspot.com/2013/06/submitting-hadoop-jobs-programmatically.html

Moreover, I develop on Windows (don't ask me why), so one more thing to worry about since Hadoop has not been developed with that OS in mind the first place.

Regards,
Vjeran